一次性在SQL Server 2008中插入1000条以上的记录

Question

我正在创建一个WPF应用程序，客户端可以在其中将多个记录插入数据库中。

插入记录的最小限制为1000。

我使用下面的代码在c＃中插入了相同的代码。

var response = SearchEngine.Search(objSearchRequest);

if (response.ResponseStatus == SearchResponse.SearchResponseStatus.Success)
{
    ImageClippingQueue objImageClippingQueue;

    if (response.SearchResults.Count > 0)
    {
       foreach (var item in response.SearchResults)
       {
          objImageClippingQueue = new ImageClippingQueue();
          objImageClippingQueue.ImageID = item.Value.ImageId;
          objImageClippingQueue.pubid = item.Value.PubId;

          new CommonMethods().InsertImageData(objImageClippingQueue);
       }

       UpdateClippingSearchKeyword(keywordId, response.TotalAvailableResults, (int)totalRecordsImported);
    }
}

此方法InsertImageData将数据插入数据库。

有没有什么最快的方法可以使数据库命中一次而不是一次又一次命中。

谢谢

Answer 1

您可以使用SqlBulkCopy类插入数据。 以下示例摘自http://msdn.microsoft.com/zh-cn/library/system.data.sqlclient.sqlbulkcopy(v=vs.110).aspx

            using (SqlBulkCopy bulkCopy =
                       new SqlBulkCopy(destinationConnection))
            {
                bulkCopy.DestinationTableName =
                    "dbo.BulkCopyDemoMatchingColumns";

                try
                {
                    // Write from the source to the destination.
                    bulkCopy.WriteToServer(reader);
                }
                catch (Exception ex)
                {
                    Console.WriteLine(ex.Message);
                }
                finally
                {
                    // Close the SqlDataReader. The SqlBulkCopy 
                    // object is automatically closed at the end 
                    // of the using block.
                    reader.Close();
                }
            }

Answer 2

我做了类似以下的事情：

public static void insert(List<myRecord> records)
{
    string sql = "insert into myTable (col1, col2) values ";

    int i = 0;
    foreach (record in records)
    {
        if (0 < i) sql += ",";

        sql += "(@col1_" + i + ",@col2_" + i + ")";
        ++i;
    }

    SqlConnection conn = new SqlConnection(myConnStr);
    SqlCommand cmd = new SqlCommand(sql, conn);

    i = 0;
    foreach (record in records)
    {
        cmd.Parameters.Add("@col1_" + i, System.Data.SqlDbType.Int).Value = record.Val1;
        cmd.Parameters.Add("@col2_" + i, System.Data.SqlDbType.Int).Value = record.Val2;
        ++i;
    }

    // and so on...
}

我还不得不将其限制为每个插入仅插入500条记录，但这对客户端是隐藏的。 使用StringBuilder也将有所改进。

Answer 3

我使用存储过程来完成此任务。

链接： http ： //programcall.com/11/dotnet/how-to-read-data-from-xml-string--and-insert-to-table-sql-server.aspx

-- To import multiple records into database
Create PROCEDURE SP_Importimagedata (
 @xmlData XML 
)

AS
BEGIN

    INSERT INTO  [imagedata](
    ImageID,
    ImageClippingStatusID,
    [Priority],
    UserID,
    pubid,
    ClipTypeID
    )

    SELECT
    COALESCE([Table].[Column].value('ImageID[1]', 'int'),0) as 'ImageID',
    [Table].[Column].value('ImageClippingStatusID[1]', 'int') as 'ImageClippingStatusID',
    [Table].[Column].value(' Priority[1]', 'int') as 'Priority',
    [Table].[Column].value(' UserID[1]', 'int') as 'UserID',
    [Table].[Column].value(' pubid[1]', 'int') as 'pubid',
    [Table].[Column].value(' ClipTypeID[1]', 'int') as 'ClipTypeID'
    FROM @xmlData.nodes('/Images/Image') as [Table]([Column])
END


 var response = SearchEngine.Search(objSearchRequest);

            if (response.ResponseStatus == SearchResponse.SearchResponseStatus.Success)
            {
                if (response.SearchResults.Count > 0)
                {
                    sbCreateDataXml = new StringBuilder();
                    sbCreateDataXml.Append("<Images>");
                    foreach (var item in response.SearchResults)
                    {
                        sbCreateDataXml.Append("<Image>");
                        sbCreateDataXml.Append("<ImageID>" + item.Value.ImageId + "</ImageID>");
                        sbCreateDataXml.Append("<ImageClippingStatusID>" + imageClippingStatusID + "</ImageClippingStatusID>");
                        sbCreateDataXml.Append("<Priority>" + priority + "</Priority>");
                        sbCreateDataXml.Append("<UserID>" + userID + "</UserID>");
                        sbCreateDataXml.Append("<pubid>" + item.Value.PubId + "</pubid>");
                        sbCreateDataXml.Append("<ClipTypeID>" + clipTypeID + "</ClipTypeID>");
                        sbCreateDataXml.Append("</Image>");
                        counter++;
                    }
                    sbCreateDataXml.Append("</Images>");
                    new CommonMethods().InsertImageClippingQueue(sbCreateDataXml.ToString());
                    totalRecordsImported = (int)totalRecordsImported + counter;
                    UpdateClippingSearchKeyword(keywordId, response.TotalAvailableResults, (int)totalRecordsImported);
                }
            }

它的效果更好。

请让我知道我们是否可以做得更好。

谢谢。

Answer 4

您可以使用sql批量插入：

DataTable table = new DataTable("States");
// construct DataTable
table.Columns.Add(new DataColumn("id_state", typeof(int))); 
table.Columns.Add(new DataColumn("state_name", typeof(string)));

// note: if "id_state" is defined as an identity column in your DB,
// row values for that column will be ignored during the bulk copy
table.Rows.Add("1", "Atlanta");
table.Rows.Add("2", "Chicago");
table.Rows.Add("3", "Springfield");

using(SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString))
{
  bulkCopy.BulkCopyTimeout = 600; // in seconds
  bulkCopy.DestinationTableName = "state";
  bulkCopy.WriteToServer(table);
}