Entity Framework Core - 同时运行的查询在不同的环境中表现不同

Question

Problem : I have noticed that Entity Framework calls started at the same time are getting incrementally longer to process. The program is .NET Core 2.2, with EF 2.2

Here is an example of logs that demonstrate the behavior:

and it goes on to up to 10 seconds at the 30th query.

Investigation:

To reproduce and test this, I have created a simple executable that will call a method Test() 20x times in parallel, Test() is getting the scoped context from ServiceProvider, and doing a query to the database. I have executed it few times and showing a screenshot from 3 results on each setup. Note: the Query() method is run once before the 20 tests, allowing EF to generate the query cache

 private async Task Test()
    {
        List<Task> tasks = new List<Task>();
        // Populate the queue.
        for (int i = 0; i < 10000; i++)
        {
            cq.Enqueue(i);
        }
        Query();
        Thread.Sleep(1000);
        Query();
        Thread.Sleep(1000);
        for (int i = 0; i < 20; i++)
        {
            tasks.Add(Task.Run(() => Query()));
        }


        await Task.WhenAll(tasks);

    }



    private async Task Query()
    {
        cq.TryDequeue(out int id);

        Log.Debug($"[{id}] {DateTime.Now.ToString("mm:ss.fff")}: started");

        using (var serviceScope = _serviceProvider.CreateScope())
        {
            var watch = new Stopwatch();
            var ctx = serviceScope.ServiceProvider.GetService<Core.Models.MyContext>();

            watch.Start();

            var quoteLineJob = await ctx.QuoteLineJobs
                .FirstOrDefaultAsync(o => o.Id == id);

            watch.Stop();

            Log.Debug($"[{id}] {DateTime.Now.ToString("mm:ss.fff")}: Quote line job loaded in {watch.ElapsedMilliseconds}ms");
            
        }
    }

I can't make sense of the results and hope someone could suggest a hypothesis:

• From Production Client machine to Production DB

the loading is incrementally longer

• From developer machine to Production DB

Hitting the same database, however, results are much slower.

• Random Server to Production DB

The result is pretty similar as from the developer's machine, however, 50% chance that it behaves randomly (See 3rd result) and not incrementally

I would like to add that looking at a SQL Server Profiler, for all setup, the read queries take 0ms for all simultaneous queries.

As if this wasn't confusing, when I tried the same test on localhost:

• Developer's machine to developer's DB (DB Copy from production, same data)

results here are random and not incremented

• Random Server to Random Server DB

Again, loading times are random and not incremented

Does anyone see some patterns, and suggestions as to why Entity Framework would behave like this?

Answer 1

My first suggestion would be to determine whether the cause is EF or your service wrapper, depending on how that is implemented. Change your test to this instead and check the performance on the different environments:

var watch = new Stopwatch();
watch.Start();
using (var ctx = new YourAppDbContext(ConnectionString))
{
    var quoteLineJob = await ctx.QuoteLineJobs
        .FirstOrDefaultAsync(o => o.Id == id);
}
watch.Stop();

Log.Debug($"[{id}] {DateTime.Now.ToString("mm:ss.fff")}: Quote line job loaded in {watch.ElapsedMilliseconds}ms");
        

If this behaves more predictably then something is suss with your context scope or provider for the DbContext. (unlikely, but without seeing the code, a possibility)

If it is still somewhat variable this is probably a symptom of parallelization. While executing async code, the awaited methods will be setting resume execution points and freeing up the executing thread. .Net will be responding to requests to resume thread operations. Given these operations are already being parallelized into tasks one test would be to make the EF calls synchronously to assess whether more predictable performance comes back. The server's # of cores and overall load situation could greatly effect how 20 or more parallelized are executed.

var watch = new Stopwatch();
watch.Start();
using (var ctx = new YourAppDbContext(ConnectionString))
{
    var quoteLineJob = ctx.QuoteLineJobs
        .FirstOrDefault(o => o.Id == id);
}
watch.Stop();

Log.Debug($"[{id}] {DateTime.Now.ToString("mm:ss.fff")}: Quote line job loaded in {watch.ElapsedMilliseconds}ms");

Again, this would just be to help identify the potential cause of performance quandaries. async / await do not equal better performance, they make individual operations slower but make the server as a whole more responsive to handle more requests without waiting on slower or high-frequency tasks to complete before picking up a request. The context switching between threads will mean each task will operate slower and potentially variably so when the server is under load. I would reserve async operations to tasks that are expected to take a considerable amount of time or expected to be called extremely frequently.