How to Limit request per Second in Async Task C#

Question

I'm writing an application which interact with Azure Cosmos DB. I need commit 30,000 Records to CosmosDB in a Session. Because I used .NET Core so I cannot use BulkInsert dll. So, I use Foreach loop to Insert to CosmosDB. But I see too many request per Second and it overload RU limit by CosmosDB.

foreach(item in listNeedInsert){
      await RequestInsertToCosmosDB(item);
}

I want to Pause foreach loop when Number of request reach 100. After done 100 request. foreach will continue.

Answer 1

You can partition the list and await the results:

var tasks = new List<Task>();

foreach(item in listNeedInsert)
{
    var task = RequestInsertToCosmosDB(item);
    tasks.Add(task);

    if(tasks.Count == 100)
    {
        await Task.WhenAll(tasks);
        tasks.Clear();
    }
}

// Wait for anything left to finish
await Task.WhenAll(tasks);

Every time you've got 100 tasks running the code will wait for them all to finish before executing the last batch.

Answer 2

You could set a delay on every hundred iteration

int i = 1;
foreach(item in listNeedInsert)
{
      await RequestInsertToCosmosDB(item);
      if (i % 100 == 0)
      {
          i = 0;
          await Task.Delay(100); // Miliseconds
      }
      i++;
}

Answer 3

If you really want to maximize efficiency and can't do bulk updates, look into using SemaphorSlim in this post:

Throttling asynchronous tasks

Hammering a medium-sized database with 100 concurrent requests at a time isn't a great idea because it's not equipped to handle that kind of throughput. You could try playing with a different throttling number and seeing what's optimal, but I would guess it's in the single digit range.

If you want to do something quick and dirty, you could probably use Sean's solution. But I'd set the Task count to 5 starting out, not 100.

Answer 4

https://github.com/thomhurst/EnumerableAsyncProcessor

I've written a library to help with this sort of logic.

Usage would be:

await AsyncProcessorBuilder.WithItems(listNeedInsert) // Or Extension Method: listNeedInsert.ToAsyncProcessorBuilder()
        .ForEachAsync(item => RequestInsertToCosmosDB(item), CancellationToken.None)
        .ProcessInBatches(batchSize: 100);