[英]Retryable SQLBulkCopy for SQL Server 2008 R2
I am basically from database background and new to .net stuff. 我基本上是从数据库背景和.net的新手。 Please bear with me if my question sounds silly. 如果我的问题听起来很傻,请忍受我。
I am using SqlBulkCopy in my code to transfer data from one SQL Server to other. 我在代码中使用SqlBulkCopy将数据从一个SQL Server传输到另一个。 But it is failing frequently due to network issues. 但是由于网络问题,它经常失败。 To avoid that I am planning to do two things 为了避免这种情况,我打算做两件事
Decrease batch size (from 5000 to 1000) and increase timeout (from 3min. to 1min) 减小批量大小(从5000到1000)并增加超时(从3分钟到1分钟)
Implement retry logic 实施重试逻辑
My question is 我的问题是
Sample Code that I am using: 我正在使用的示例代码:
private void BulkCopyTable(string schemaName, string tableName)
{using (var reader = srcConnection.ExecuteReader($"select * from [{SourceDBName}].[{schemaName}].[{tableName}]"))
{
const SqlBulkCopyOptions bulkCopyOptions = SqlBulkCopyOptions.TableLock | SqlBulkCopyOptions.FireTriggers |
SqlBulkCopyOptions.KeepNulls |
SqlBulkCopyOptions.KeepIdentity;
using (var bcp = new SqlBulkCopy(dstConnection.ConnectionString, bulkCopyOptions))
{
const int threeMinutes = 60*3;
bcp.BulkCopyTimeout = threeMinutes; //Timeout is for a single batch
bcp.BatchSize = 5000;
bcp.DestinationTableName = $"[{DestinationDB}].[{schemaName}].[{tableName}]";
bcp.EnableStreaming = true;
foreach (var col in table.Columns.Cast<Column>().Where(c => !c.Computed))
{
bcp.ColumnMappings.Add(col.Name, col.Name);
}
bcp.WriteToServer(reader);
}
}
}
A simple approach is to: 一种简单的方法是:
SqlBulkCopy
needs to query meta data for each WriteToServer
call. 由于SqlBulkCopy
需要为每个WriteToServer
调用查询元数据,因此这会导致效率低下。 So don't make the batches too small. 因此,不要使批次过小。 Experiment. 实验。 #temp
table but a durable one so that you can lose the connection and continue). 插入一个临时表(不是#temp
表,而是一个持久的表,这样就可以断开连接并继续)。 insert...select
as the final step to move the rows from the temp table to the real table. 然后,执行insert...select
作为最后一步,将行从temp表移动到实表。 This dance splits the work into retryable batches but acts as if it was one transaction. 这种舞蹈将作品分成多个可重试的批次,但就像是一笔交易一样。
If you don't need atomicity you can leave it at step (1). 如果不需要原子性,可以将其保留在步骤(1)中。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.