How do I use Java to read a .csv file and insert its data into SQL Server?

Question

I am very new to Java. I have a task with two steps.

1. I want to read all data from a .csv file.
2. After reading that data I have to put it into a SQL Server database.

I have done step one. I'm able to read the .csv file data, but I don't know that how to insert it into a database.

Here's my code for fetching the .csv file data:

import java.io.BufferedReader;  
import java.io.FileNotFoundException;  
import java.io.FileReader;  
import java.io.IOException;  
import java.util.StringTokenizer;  


public class DBcvsdataextractor {  

    /** 
     * @param args 
    */  
    public static void main(String[] args) { 
        // TODO Auto-generated method stub  

        String fileName="D:/USPresident Wikipedia URLs Thumbs HS.csv";  

        try {    
        BufferedReader br = new BufferedReader( new FileReader(fileName));    
            StringTokenizer st = null;    
            int lineNumber = 0, tokenNumber = 0;    

            while( (fileName = br.readLine()) != null)    
            {    

                if(lineNumber++ == 0)  
                   continue;                  

                //break comma separated line using ","    
                st = new StringTokenizer(fileName, ",");    

                while(st.hasMoreTokens())    
                {    
                    //display csv values    
                    tokenNumber++;    
                    System.out.print(st.nextToken() + '\t'); 
                }    

                //new line    
                System.out.println(" ");    

                //reset token number    
                tokenNumber = 0;    

            }    
        } catch (FileNotFoundException e) {    
            // TODO Auto-generated catch block    
            e.printStackTrace();    
        } catch (IOException e) {    
            // TODO Auto-generated catch block    
            e.printStackTrace();    
        }
    }
}

Now, how do I insert that data into SQL Server?

Answer 1

The SQL Server has a tool for this.

Like this:

BULK INSERT CSVTest
FROM 'c:\csvtest.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)

Use this command in your java, using JDBC connection, for example.

Hope this helps.

SQL Server file CSV

Answer 2

If you really want to do this with Java, and really want to write your own CSV parser as you currently did, you can

1. Instead of printing out each 'CSV-file value', you will have to store them. You could for example use an ArrayList for each column in the CSV file, and populate those while reading the CSV file
2. Once the file is read, you can loop over those ArrayList instances again to construct one big INSERT statement for all data, or one INSERT statement for each row you encountered in the CSV file. If you would opt for the last option, it is not even necessary to use those ArrayList instances. In that case you could construct an indiviual INSERT statement while reading the CSV file, and submit it to the DB after each time a row has been read.

I know the approach of constructing your query while reading the CSV file would be possible as well if you want to go for one big INSERT statement, but separating the INSERT from the reading of the CSV file has the big advantage you can replace your own CSV parser later on by a standard one without too much trouble.

Answer 3

You can customize below class to insert data into sql server.

File ImportCsv.java

package com.example.demo;

import java.io.InputStreamReader;
import java.io.Reader;
import java.io.UnsupportedEncodingException;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.io.FileInputStream;
import java.io.FileNotFoundException;

import com.opencsv.CSVReader;

public class ImportCsv
{
    public static void main(String[] args) throws UnsupportedEncodingException, FileNotFoundException
    {
            readCsv();
    }

    private static void readCsv() throws UnsupportedEncodingException, FileNotFoundException
    {
        Reader readerstream = new InputStreamReader(new FileInputStream("D:\\file.csv"), "Unicode");
            try (CSVReader reader = new CSVReader(readerstream, ',');
                    
                    Connection connection = DBConnection.getConnection();)
            {
                    String insertQuery = "Insert into [dbo].[tableName] ([Column1],[Column2],[Column3], [Column4],...[Column8]") values (?,?,?,?,?,?,?,?)";
                    PreparedStatement pstmt = connection.prepareStatement(insertQuery);
                    
                    String[] rowData = null;
                    int i = 0;
                    while((rowData = reader.readNext()) != null){
                    for (String data : rowData)
                    {
                        //System.out.println(new String(data.replace(" ","")));
                        String strLine = new String(data.replace(" ",""));
                        String[] splited = strLine.split("\\s+");
                        

                            pstmt.setNString(1, splited[0]);
                            pstmt.setNString(2, splited[1]);
                            pstmt.setNString(3, splited[2]);
                            pstmt.setNString(4, splited[3]);
                            pstmt.setNString(5, splited[4]);
                            pstmt.setNString(6, splited[5]);
                            pstmt.setNString(7, splited[6]);
                            try {
                            pstmt.setNString(8, splited[7]);
                            }catch(Exception e) {
                                
                            }
                            if (++i % 8 == 0) {
                                    pstmt.addBatch();// add batch
                            }
                            if (i % 80 == 0) {// insert when the batch size is 10
                                    pstmt.executeBatch();
                            }
                    }}
                    System.out.println("Data Successfully Uploaded");
            }
            catch (Exception e)
            {
                    e.printStackTrace();
            }

    }
  }

DBConnection.java

package com.example.demo;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

public class DBConnection {

static {

    try {
        Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
    } catch (Exception e) {
        e.printStackTrace();
    }
}

public static Connection getConnection() throws SQLException {
 String url = "jdbc:sqlserver://localhost:1433;" +
      "databasename=myDBName;user=user;password=pwd;sendStringParametersAsUnicode=true;";    
 Connection con =DriverManager.getConnection(url);
    return con;

}


}