Timeout in LINQ to SQL inserting millions of records

Question

I'm inserting approximently 3 million records in a database using this solution . Eventually when the application has been inserting records for a while (my last run lasted around 4 hours), it gives a timeout with the following SqlException:

"SqlExcepetion: Timeout expired. The timeoutperiod elapsed prior to completion of the operation or the server is not responding."

What's the best way to handle this exception? Is there a way to prevent this from happening or should I catch the exception?

Thanks in advance!

Answer 1

If everything you have is a hammer, every problem looks like a nail.

Seriously, even TRYING to use Linq2SQL for inserting 3 million records is a broken approach. ORM's are a lot of good things, they are NOT bulk insert elements.

I suggest:

• Generate the table data into a file
• Load the file using appropriate tools (bulk load mechanism). You wil lbe surprised about the performance (hint: I load about 600 million records in 15 minutes);)

Bulk load has a lot of performance advantages. Naturally assuming you talk of something like an ETL load process here - but I fail to see a 3 million load as anything except a data warehouse / reporting / archiving style of load operation, it definitlely and per definition is not real time transaction processing ;)

Use the right tool for the job.

Answer 2

Generate a SQL script (or string), and execute that directly.

L2SQL is not made for mass operations like this.

Answer 3

I think you've to increase the CommandTimeout of your command.

Solution can be found here