SQL Bulk Insert in C# not inserting values

Question

I'm completely new to C#, so I'm sure I'm going to get a lot of comments about how my code is formatted - I welcome them. Please feel free to throw any advice or constructive criticisms you might have along the way.

I'm building a very simple Windows Form App that is eventually supposed to take data from an Excel file of varying size, potentially several times per day, and insert it into a table in SQL Server 2005. Thereafter, a stored procedure within the database takes over to perform various update and insert tasks depending on the values inserted into this table.

For this reason, I've decided to use the SQL Bulk Insert method, since I can't know if the user will only insert 10 rows - or 10,000 - at any given execution.

The function I'm using looks like this:

public void BulkImportFromExcel(string excelFilePath)
{
    excelApp = new Excel.Application();
    excelBook = excelApp.Workbooks.Open(excelFilePath);
    excelSheet = excelBook.Worksheets.get_Item(sheetName);
    excelRange = excelSheet.UsedRange;
    excelBook.Close(0);
    try
    {
        using (SqlConnection sqlConn = new SqlConnection())
        {
            sqlConn.ConnectionString =
            "Data Source=" + serverName + ";" +
            "Initial Catalog=" + dbName + ";" +
            "User id=" + dbUserName + ";" +
            "Password=" + dbPassword + ";";
            using (OleDbConnection excelConn = new OleDbConnection())
            {
                excelQuery = "SELECT InvLakNo FROM [" + sheetName + "$]";
                excelConn.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + excelFilePath + ";Extended Properties='Excel 8.0;HDR=Yes'";
                excelConn.Open();
                using (OleDbCommand oleDBCmd = new OleDbCommand(excelQuery, excelConn))
                {
                    OleDbDataReader dataReader = oleDBCmd.ExecuteReader();
                    using (SqlBulkCopy bulkImport = new SqlBulkCopy(sqlConn.ConnectionString))
                    {
                        bulkImport.DestinationTableName = sqlTable;
                        SqlBulkCopyColumnMapping InvLakNo = new SqlBulkCopyColumnMapping("InvLakNo", "InvLakNo");
                        bulkImport.ColumnMappings.Add(InvLakNo);
                        sqlQuery = "IF OBJECT_ID('ImportFromExcel') IS NOT NULL BEGIN SELECT * INTO [" + DateTime.Now.ToString().Replace(" ", "_") + "_ImportFromExcel] FROM ImportFromExcel; DROP TABLE ImportFromExcel; END CREATE TABLE ImportFromExcel (InvLakNo INT);";
                        using (SqlCommand sqlCmd = new SqlCommand(sqlQuery, sqlConn))
                        {
                            sqlConn.Open();
                            sqlCmd.ExecuteNonQuery();
                            while (dataReader.Read())
                            {
                                bulkImport.WriteToServer(dataReader);
                            }
                        }
                    }
                }
            }
        }
    }
    catch(Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
    finally
    {
        excelApp.Quit();
    }
}

The function runs without errors or warnings, and if I replace the WriteToServer with manual SQL commands, the rows are inserted; but the bulkImport isn't inserting anything.

NOTE: There is only one field in this example, and in the actual function I'm currently running to test; but in the end there will be dozens and dozens of fields being inserted, and I'll be doing a ColumnMapping for all of them.

Also, as stated, I am aware that my code is probably horrible - please feel free to give me any pointers you deem helpful. I'm ready and willing to learn.

Thanks!

Answer 1

I think it would be a very long and messy answer if I commented on your code and also gave pointer sample codes in the same message, so I decided to divide then into two messages. Comments first:

You are using automation to get what? You already have the sheet name as I see it and worse you are doing app.Quit() at the end. Completely remove that automation code. If you needed some information from excel (like sheet names, column names) then you could use OleDbConnecton's GetOleDbSchemaTable method. You might do the mapping basically in 2 ways:

1. Excel column ordinal to SQL table column name
2. Excel column name to SQL table column name

both would do. In a generic code, assuming you have column names same in both sources, but their ordinal and count may differ, you could get the column names from OleDbConnection schema table and do the mapping in a loop.

You are dropping and creating a table named "ImportFromExcel" for the purpose of temp data insertion, then why not simply create a temp SQL server table by using a # prefix in table name? OTOH that code piece is a little weird, it would do an import from "ImportFromExcel" if it is there, then drop and create a new one and attempt to do bulk import into that new one. In first run, SqlBulkCopy (SBC) would fill ImportFromExcel and on next run it would be copied to a table named (DateTime.Now ...) and then emptied via drop and create again. BTW, naming:

DateTime.Now.ToString().Replace(" ", "_") + "_ImportFromExcel"

doesn't feel right. While it looks tempting, it is not sortable, probably you would want something like this instead:

DateTime.Now.ToString("yyyyMMddHHmmss") + "_ImportFromExcel"

Or better yet:

"ImportFromExcel_" +DateTime.Now.ToString("yyyyMMddHHmmss")

so you would have something that is sorted and selectable for all the imports as a wildcard or looping for some reason.

Then you are writing to server inside a reader.Read() loop. That is not the way WriteToServer works. You wouldn't do reader.Read() but simply:

sbc.WriteToServer(reader);

In my next message e I will give simple schema reading and a simple SBC sample from excel into a temp table, as well as a suggestion how you should do that instead.

Answer 2

the WriteToServer(IDataReader) is intended to do internally the IDataReader.Read() operation.

using (SqlCommand sqlCmd = new SqlCommand(sqlQuery, sqlConn))
{
    sqlConn.Open();
    sqlCmd.ExecuteNonQuery();
    bulkImport.WriteToServer(dataReader);
}

You can check the MSDN doc on that function, has a working example: https://msdn.microsoft.com/en-us/library/434atets(v=vs.110).aspx

Answer 3

Here is the sample for reading schema information from Excel (here we read the tablenames - sheet names with tables in them):

private IEnumerable<string> GetTablesFromExcel(string dataSource)
{
    IEnumerable<string> tables;
    using (OleDbConnection con = new OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;" +
    string.Format("Data Source={0};", dataSource) +
    "Extended Properties=\"Excel 12.0;HDR=Yes\""))
    {
        con.Open();
        var schemaTable = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
        tables = schemaTable.AsEnumerable().Select(t => t.Field<string>("TABLE_NAME")); 
        con.Close();
    }
    return tables;
}

And here is a sample that does SBC from excel into a temp table:

void Main()
{
  string sqlConnectionString = @"server=.\SQLExpress;Trusted_Connection=yes;Database=Test";

  string path = @"C:\Users\Cetin\Documents\ExcelFill.xlsx"; // sample excel sheet
  string sheetName = "Sheet1$";

  using (OleDbConnection cn = new OleDbConnection(
    "Provider=Microsoft.ACE.OLEDB.12.0;Data Source="+path+
    ";Extended Properties=\"Excel 8.0;HDR=Yes\""))

  using (SqlConnection scn = new SqlConnection( sqlConnectionString ))
  {

    scn.Open();
    // create temp SQL server table
    new SqlCommand(@"create table #ExcelData 
    (
      [Id] int, 
      [Barkod] varchar(20)
    )", scn).ExecuteNonQuery();

    // get data from Excel and write to server via SBC  
    OleDbCommand cmd = new OleDbCommand(String.Format("select * from [{0}]",sheetName), cn);
    SqlBulkCopy sbc = new SqlBulkCopy(scn);

    // Mapping sample using column ordinals
    sbc.ColumnMappings.Add(0,"[Id]");
    sbc.ColumnMappings.Add(1,"[Barkod]");

    cn.Open();
    OleDbDataReader rdr = cmd.ExecuteReader();
    // SqlBulkCopy properties
    sbc.DestinationTableName = "#ExcelData";
    // write to server via reader
    sbc.WriteToServer(rdr);
    if (!rdr.IsClosed) { rdr.Close(); }
    cn.Close();

    // Excel data is now in SQL server temp table
    // It might be used to do any internal insert/update 
    // i.e.: Select into myTable+DateTime.Now
    new SqlCommand(string.Format(@"select * into [{0}] 
                from [#ExcelData]", 
                "ImportFromExcel_" +DateTime.Now.ToString("yyyyMMddHHmmss")),scn)
        .ExecuteNonQuery();
    scn.Close();
  }
}

While this would work, thinking in the long run, you need column names, and maybe their types differ, it might be an overkill to do this stuff using SBC and you might instead directly do it from MS SQL server's OpenQuery:

SELECT * into ... from OpenQuery(...)