实体框架4.0批量操作的性能

Question

I'm currently in the process of improving the performance of an existing c# project that uses entity framework (and has 4.0 version). There are 2 types of bulk operations done in the application:

1. Bulk inserts
2. Bulk deletes

Currently they are done using pure SQL statements ("INSERT INTO...", "DELETE FROM...") (the insert statement itself is currently no bulk insert statement, but a "normal" insert statement instead).

As I'm pretty new to C# my first step was to look around how the performance would be if I use the entity framework for the updates and deletes.

My question here is three folded:

1. Is it true that if I try to do a bulk insert with the entity framework that it does use 1 insert per data row inserted? (thus a roundtrip for each insert). And thus that the performance is less than using an SQL "insert"?
2. Does this also hold true for delete statements?
3. What is the best practice here? To use SQL-statements? Or to use the entity framework, or something else entirely?

(for the data rows I'm talking about sizes of 2k-200k each).

Thanks

Answer 1

My question here is three folded:

Is it true that if I try to do a bulk insert with the entity framework that it does use 1 insert per data row inserted? (thus a roundtrip for each insert). And thus that the performance is less than using an SQL "insert"?

Yes. If you want to use EF set the following properties to false to get faster performance:

MyContext.Configuration.AutoDetectChangesEnabled = false;
MyContext.Configuration.ValidateOnSaveEnabled = false;

Does this also hold true for delete statements?

Yes. Also you can define on the database On Delete Cascade then the database will delete the referenced entities so there is no need to do this using EF.

What is the best practice here? To use SQL-statements? Or to use the entity framework, or something else entirely?

You can use stored procedure, call Query on your Context

MyContext.ExecuteStoreQuery("your query")

or

MyContext.Database.SqlCommand("your query"); 

The other approach is to call SaveChanges() after a batch (100, 200 entities marked as Added or Deleted) and then dispose the context so that the entities aren't still attached. Then create a new context make a batch and call SaveChanges() again.

UPDATE

I didn't use this approach but you can try it out.

SqlBulkCopy for Generic List (useful for Entity Framework & NHibernate)

Reusable generic version below, which produced 15k inserts in 2.4s or +- 6200 rows per second. I upped it to 4 catalogs, 224392 rows in 39s, for +- 5750 rps (changing between 4 files).

Answer 2

If you truly want to insert bulk data you probably want to use SqlBulkCopy . You can use it in the same transaction that your EF context uses.

EF is not made for bulk operations and you might find its single-row-per-statement DML approach to be too restrictive for large sets. It forces lots of round-trips, lots of per-statement overheads and prevents SQL Server from optimizing a query plan for many rows at once which is almost always more efficient than many small queries (for example SQL Server will properly sort all the rows so that indexes can be updates sequentially).

By using EF to do bulk DML you basically force SQL Server to use per-row DML plans.

Bulk deletes can be handled by bulk-inserting the keys into a temp table and then executing a delete statement joining to that table.