Number of query values and destination fields are not the same - C# Script Task SSIS - Export Results of SQL Proc to Excel Using Dynamic Columns

Question

I am trying to write a script task in SSIS that runs a SQL procedure based on a variable, determines the name and number of columns from the results, then exports that into an excel sheet. This is based off of the TechBrothersIT site/video: http://www.techbrothersit.com/2016/03/how-to-create-excel-file-dynamically_21.html

The error "System.Data.OleDb.OleDbException (0x80040E14): Number of query values and destination fields are not the same." occurs after it runs the procedure, as it tries to insert into the table it builds. I've tried to search for an answer but I have yet to see anyone use dynamic columns like this, so all of the answers saying to specify your columns for the INSERT and VALUES portions don't apply (from what I can tell).

The procedure being run is returning six columns: Branch, Client, Hours, Gross Pay, WC Premium, GP$. Using messagebox.show after it dynamically builds to command to build the table confirms it is using these values, with the correct spelling, and encased in brackets. Because of that, I believe my problem is in the VALUES portion, but I can't figure it out.

What am I doing wrong here?

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;
using System.Data.OleDb;
using System.Data.SqlClient;

namespace ST_0493a2bda5424767ac07ca96649a95e2
{
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        public void Main()
        {
            string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
            try
            {
                //Declare Variables
                string ExcelFileName = Dts.Variables["User::ExcelFileName"].Value.ToString();
                string FolderPath = Dts.Variables["User::FolderPath"].Value.ToString();
                string StoredProcedureName = Dts.Variables["User::StoredProcedureName"].Value.ToString();
                string SheetName = Dts.Variables["User::SheetName"].Value.ToString();
                string StartDate = Dts.Variables["User::StartDate"].Value.ToString();
                string EndDate = Dts.Variables["User::EndDate"].Value.ToString();
                string DBName = Dts.Variables["User::DBName"].Value.ToString();
                string ServerName = Dts.Variables["User::ServerName"].Value.ToString();
                ExcelFileName = ExcelFileName + "_" + datetime;

                OleDbConnection Excel_OLE_Con = new OleDbConnection();
                OleDbCommand Excel_OLE_Cmd = new OleDbCommand();

                //Construct ConnectionString for Excel
                string connstring = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + FolderPath + ExcelFileName
                    + ".xlsx;Extended Properties=\"Excel 12.0 Xml;HDR=YES\";";

                //Drop Excel file if exists
                File.Delete(FolderPath + "\\" + ExcelFileName + ".xlsx");

                //USE ADO.NET Connection from SSIS Package to get data from table
                SqlConnection myADONETConnection = new SqlConnection("Data Source="+ServerName+";Initial Catalog="+DBName+";Integrated Security=true");                


                //Load Data into DataTable from SQL ServerTable
                string queryString = "EXEC  " + StoredProcedureName + " '" + StartDate + "', '" + EndDate + "', @RunInSSIS=1";
                SqlDataAdapter adapter = new SqlDataAdapter(queryString, myADONETConnection);
                DataSet ds = new DataSet();
                adapter.Fill(ds);

                //MessageBox.Show(queryString);

                //Get Header Columns
                string TableColumns = "";

                // Get the Column List from Data Table so can create Excel Sheet with Header
                foreach (DataTable table in ds.Tables)
                {
                    foreach (DataColumn column in table.Columns)
                    {
                        TableColumns += column + "],[";
                    }
                }

                // Replace most right comma from Columnlist
                TableColumns = ("[" + TableColumns.Replace(",", " NVARCHAR(255),").TrimEnd(','));
                TableColumns = TableColumns.Remove(TableColumns.Length - 2);
                MessageBox.Show(TableColumns);


                //Use OLE DB Connection and Create Excel Sheet
                Excel_OLE_Con.ConnectionString = connstring;
                Excel_OLE_Con.Open();
                Excel_OLE_Cmd.Connection = Excel_OLE_Con;
                Excel_OLE_Cmd.CommandText = "Create table " + SheetName + " (" + TableColumns + ")";
                Excel_OLE_Cmd.ExecuteNonQuery();


                //Write Data to Excel Sheet from DataTable Dynamically
                foreach (DataTable table in ds.Tables)
                {
                    String sqlCommandInsert = "";
                    String sqlCommandValue = "";
                    foreach (DataColumn dataColumn in table.Columns)
                    {
                        sqlCommandValue += dataColumn + "],[";
                    }

                    sqlCommandValue = "[" + sqlCommandValue.TrimEnd(',');
                    sqlCommandValue = sqlCommandValue.Remove(sqlCommandValue.Length - 2);
                    sqlCommandInsert = "INSERT into " + SheetName + "(" + sqlCommandValue + ") VALUES(";

                    int columnCount = table.Columns.Count;
                    foreach (DataRow row in table.Rows)
                    {
                        string columnvalues = "";
                        for (int i = 0; i < columnCount; i++)
                        {
                            int index = table.Rows.IndexOf(row);

                            columnvalues += table.Rows[index].ItemArray[i];
                            columnvalues = "'" + columnvalues.Replace("'", "''") + "',";
                        }
                        columnvalues = columnvalues.TrimEnd(',');

                        var command = sqlCommandInsert + columnvalues + ")";
                        Excel_OLE_Cmd.CommandText = command;
                        Excel_OLE_Cmd.ExecuteNonQuery();
                    }

                }
                Excel_OLE_Con.Close();
                Dts.TaskResult = (int)ScriptResults.Success;
            }
            catch (Exception exception)
            {

                // Create Log File for Errors
                using (StreamWriter sw = File.CreateText(Dts.Variables["User::FolderPath"].Value.ToString() + "\\" +
                    Dts.Variables["User::ExcelFileName"].Value.ToString() + datetime + ".log"))
                {
                    sw.WriteLine(exception.ToString());
                    Dts.TaskResult = (int)ScriptResults.Failure;

                }
            }
        }

        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };

    }
}

Answer 1

I figured this out. I used a messagebox.show in the inner part of the for loop to see exactly what was being built, and it was adding a single quote to the start of the statement every time it saw a new column, to where it looked like this: '''''''''[column]',[column2], etc...'

The fix involved setting the columnvalues to ' instead of a blank at the start, then modifying columnvalues to add ',' to the end in the inner loop, before finally trimming off the excess parts at the end of the completed statement. I'm guessing this can be done in a cleaner method, but it resolved my issue none the less:

foreach (DataRow row in table.Rows)
                    {
                        string columnvalues = "'";
                        for (int i = 0; i < columnCount; i++)
                        {
                            int index = table.Rows.IndexOf(row);

                            columnvalues += table.Rows[index].ItemArray[i];
                            columnvalues += "','";
                            //MessageBox.Show(columnvalues);
                        }
                        columnvalues = columnvalues.TrimEnd('\'');
                        columnvalues = columnvalues.TrimEnd(',');

                        var command = sqlCommandInsert + columnvalues + ")";
                        //MessageBox.Show(command);

                        Excel_OLE_Cmd.CommandText = command;
                        Excel_OLE_Cmd.ExecuteNonQuery();
                    }