Speed up update of 185k rows in SQL Server 2008?

Question

I have a binary file with about 185k rows in it. C# parses file in seconds. What would be the best way to update MSSQL table with that data?

What I've tried:

1. Easiest way - read binary row, parse, update table. The whole process takes around 2 days to update all the data.
2. Combine 200 update queries and send them at once to the MSSQL. In this case, update takes 8 to 10 hrs.
3. Combine 500+ queries into single one. Works faster, but drops timeout exceptions time to time, so some updates are not going through.

Any advice on how to speed up the update process?

Answer 1

不确定你真的想通过C＃来做这件事：可能想要使用BULK INSERT并给它一个文件，你的数据格式正确。

Answer 2

I would try table-valued parameter:

http://www.codeproject.com/Articles/22392/SQL-Server-2008-Table-Valued-Parameters

Answer 3

Use SqlBulkCopy (to a temporary table) followed by MERGE .

The bulk-copy method can efficiently transfer data using a "push" from a .NET client. Using BULK INSERT requires a "pull" from the server (along with the required local-file access).

Then the MERGE command (in SQL Server 2008+) can be used to insert/update/upsert the data from the temporary table to the target table according to the desired rules. Since the data is entirely in the database at this point, this operation will be as fast as can be. Using MERGE may also result in performance advantages over many individual commands, even those within the same transaction.

See also:

• Any way to SQLBulkCopy "insert or update if exists"?
• SQL Server 2008 INSERT Optimization