插入一行列表的批量插入效率

Question

I'm attemping to use bulk insert and replace it with current common insertion in my project. Some of insertion requests (BillType.Booklet) are a list with one row and others are a list with multiple row.

   public async Task CreateBill(List<BillReceiverDto> Receivers, BillType BillType)
    {
       
       var bulkList =new List<BillReceiverDto>();

        if (BillType == BillType.Booklet)
        {
            bulkList.Add(Receivers.FirstOrDefault());
        }
        else
        {
            bulkList.AddRange(Receivers);
        }

        await _dbContextProvider.GetDbContext().BulkInsertAsync(bulkList);
    }

Bulk insert have a great performance for inserting huge data, specially more than 100. It insert 5,000 entities in 75 millisecond. But Is it efficient to use bulk insert a list with one row? Is there any drawbacks such as overhead or etc...?

Answer 1

I believe the BulkInsertAsync extension uses SqlBulkCopy under the hood. In which case, I blogged some benchmarks not that long ago, that might be useful.

While I didn't focus on single row inserts, there did seem to be a cost when using SqlBulkCopy for lower numbers of rows (100) versus Table Valued Parameter approach. As the volumes ramp up, SqlBulkCopy pulls away for performance but there was a noticeable overhead for low volume. How much of an overhead? In the grand scheme of things you're probably not going to notice 10s of milliseconds.

If you're dealing with up to hundreds of rows, I'd actually recommend a Table Valued Parameter approach for performance. Larger volumes - SqlBulkCopy.

Depending on your needs/views on overheads here, I'd be tempted to check how many rows you have to insert and use the mechanism that best fits the volume. Personally, I wouldn't use SqlBulkCopy for low numbers of rows if that is a very typical scenario, because of the overhead.

Blog: https://www.sentryone.com/blog/sqlbulkcopy-vs-table-valued-parameters-bulk-loading-data-into-sql-server

Answer 2

Disclaimer : I'm the owner of Entity Framework Extensions

It depends on the library you are using Aref Hemati,

In our library, if there are 10 entities or less to insert, we directly use a SQL statement. So the SqlBulkCopy overhead is not used.

So using our library even with one row is fine but obviously optimized for hundreds and thousands of rows.