帶有映射列的 SQL 批量復制

Question

我試圖通過映射列名從一個表批量復制到另一個表，因為源和目標可能並不總是具有相同的列。

源可以有 8 列，目標可以有 10 .. 我需要映射列和批量復制。

嘗試了下面的代碼..不起作用..得到錯誤：給定的 ColumnName 'moduleid' 與數據源中的任何列都不匹配。

來源：existingtablecolumnsPresent 有[收集時間],[logtime],[moduleid],[node],[reason],[time],[timestamp],[usecaseid]

目的：dataTable.Columns有[Node],[Time],[Reason],[Moduleid],[Usecaseid]

請指教

 public static void BatchBulkCopy(DataTable dataTable, string DestinationTbl,  List<string> columnMapping,string filename)
    {
        var program = new Program();
        // Get the DataTable 
        DataTable dtInsertRows = dataTable;

        using (SqlBulkCopy sbc = new SqlBulkCopy(program.connectionStr.ToString()))
        {


            try { 
            sbc.DestinationTableName = DestinationTbl.ToLower();

            string sourceTableQuery = "Select top 1 * from " + "[" + dataTable.TableName + "]";
            DataTable dtSource = SqlHelper.ExecuteDataset(program.connectionStr.ToString(), CommandType.Text, sourceTableQuery).Tables[0];

            for (int i = 0; i < dataTable.Columns.Count; i++)
            {    //check if destination Column Exists in Source table
                if (dtSource.Columns.Cast<DataColumn>().Select(a => "[" + a.ColumnName.ToLower() + "]").Contains(dataTable.Columns[i].ToString().ToLower()))//contain method is not case sensitive
                {
                        List<string> existingtablecolumnsPresent = dtSource.Columns.Cast<DataColumn>().Select(a => "[" + a.ColumnName.ToLower() + "]").Distinct().OrderBy(t => t).ToList();

                        int sourceColumnIndex = existingtablecolumnsPresent.IndexOf(dataTable.Columns[i].ToString().ToLower());//Once column matched get its index
                    sbc.ColumnMappings.Add(dtSource.Columns[sourceColumnIndex].ToString(), dtSource.Columns[sourceColumnIndex].ToString());//give coluns name of source table rather then destination table so that it would avoid case sensitivity
                }

            }
            sbc.WriteToServer(dtInsertRows);
            sbc.Close();
        }

        catch (Exception ex)
        {
            Log.WriteLog("BatchBulkCopy" + " - " + filename, dataTable.TableName, ex.Message.ToString());

            // To move a file or folder to a new location:
            //if (File.Exists(program.sourceFile + filename))
            //    System.IO.File.Move(program.sourceFile + filename, program.failedfiles + filename);

        }

Answer 1

根據要求（創建一個包含要插入其中的列的數據表 - 將其他列排除在外。確保您遺漏的任何列在表中標記為 NULL 或具有 DEFAULT VALUE 約束（我無法向您展示如何除非你給我看你的桌子，否則不要這樣做）；

 //This first method is psuedoCode to explain how to create your datatable. You need to do it in the way that makes sense for you.
 public DataTable createDataTable(){
    List<string> excludedColumns = new List<string>();
    excludedColumns.Add("FieldToExclude");
    //...
    DataTable dt = new DataTable();
    foreach(string col in getColumns(myTable)){
         if(!excludedColumns.Contains(name)){
         DataColumn dC = new DataColumn(name,type);
         DataTable.Add(dC);
     }
     return dt;
}


 public List<string> getColumns(string tableName)
    {
        List<string> ret = new List<string>();
        using (SqlConnection conn = getConn())
        {
            conn.Open();
            using (SqlCommand com = conn.CreateCommand())
            {
                com.CommandText = "select column_Name from information_schema.COLUMNS where table_name = @tab";
                com.Parameters.AddWithValue("@tab", tableName);
                SqlDataReader read = com.ExecuteReader();
                While(read.Read()){
                ret.Add(Convert.ToString(read[0]);
            }
            conn.Close();
        }
        return ret;
    }
 //Now, you have a DataTable that has all the columns you want to insert. Map them yourself in code by adding to the appropriate column in your datatable.
 public bool isCopyInProgess = false;//not necessary - just part of my code
    public  void saveDataTable(string tableName, DataTable table)
    {
        using (SqlConnection conn = getConn())
        {
            conn.Open();
            using (var bulkCopy = new SqlBulkCopy(conn))//, SqlBulkCopyOptions.KeepIdentity))//un-comment if you want to use your own identity column
            {
                // my DataTable column names match my SQL Column names, so I simply made this loop. However if your column names don't match, just pass in which datatable name matches the SQL column name in Column Mappings
                foreach (DataColumn col in table.Columns)
                {
                    //Console.WriteLine("mapping " + col.ColumnName+" ("+does_Column_Exist(col.ColumnName,"Item").ToString()+")");
                    bulkCopy.ColumnMappings.Add(col.ColumnName, "["+col.ColumnName+"]");
                   // Console.WriteLine("ok\n");
                }
                bulkCopy.BulkCopyTimeout = 8000;
                bulkCopy.DestinationTableName = tableName;
                bulkCopy.BatchSize = 10000;
                bulkCopy.EnableStreaming = true;
                //bulkCopy.SqlRowsCopied += BulkCopy_SqlRowsCopied;
                //bulkCopy.NotifyAfter = 10000;
                isCopyInProgess = true;
               bulkCopy.WriteToServer(table);
            }
            conn.Close();
        }
    }

此外，將其用作您的 bolumn 檢查器：

 public bool does_Column_Exist(string colName,string tableName)
    {
        bool ret = false;
        using (SqlConnection conn = getConn())
        {
            conn.Open();
            using (SqlCommand com = conn.CreateCommand())
            {
                com.CommandText = "select count(*) from information_schema.COLUMNS where column_name = @col and table_name = @tab";
                com.Parameters.AddWithValue("@tab", tableName);
                com.Parameters.AddWithValue("@col", colName);
                ret = Convert.ToInt32(com.ExecuteScalar()) == 0 ? false : true; 
            }
            conn.Close();
        }
        return ret;
    }

Answer 2

為此，您需要 C# 有什么特定原因嗎？ 似乎阻力最小的路徑是使用 SQL 來完成這項工作。

INSERT INTO table2
(column_name(s))
SELECT column_name(s)
FROM table1;