Entity Framework Core first query vs subsequent query performance

Question

Recently I have been wondering that what is the reason behind drastic difference between first query and subsequent same query performance in Entity Framework? Is it because Entity Framework has some sort of internal caching which caches the query result and all the subsequent call for same query uses that cached result?

To understand the question properly, below are some tests I have ran so far

for (var i = 0; i < 20; i++)
{
    var sw = Stopwatch.StartNew();

    var departments = _sampleContext.Departments.AsNoTracking().ToList();

    sw.Stop();

    Console.WriteLine($"Run {i+1} took {sw.ElapsedMilliseconds} ms");
}

Output:

Run 1 took 2708 ms
Run 2 took 350 ms
Run 3 took 421 ms
Run 4 took 300 ms
Run 5 took 329 ms
Run 6 took 319 ms
Run 7 took 301 ms
Run 8 took 303 ms
Run 9 took 310 ms
Run 10 took 342 ms
Run 11 took 284 ms
Run 12 took 322 ms
Run 13 took 359 ms
Run 14 took 297 ms
Run 15 took 291 ms
Run 16 took 288 ms
Run 17 took 268 ms
Run 18 took 309 ms
Run 19 took 299 ms
Run 20 took 298 ms

Here you can see that there is a drastic difference between first and second run.

What could be the reason behind this?

Apart from this, I have also tried to replicate the same scenario without using Entity Framework like below

var sqlConnection = new SqlConnection(_configuration.GetSection("ConnectionStrings:DefaultConnection").Value);

sqlConnection.Open();

for (var i = 0; i < 20; i++)
{
    var sw = Stopwatch.StartNew();

    var sqlCommand = new SqlCommand("select * from Departments", sqlConnection);

    using var sqlDataReader = sqlCommand.ExecuteReader();

    if (sqlDataReader.HasRows)
    {
        while (sqlDataReader.Read())
        {
            var id = sqlDataReader.GetInt32(0);
            var name = sqlDataReader.GetString(1);
        }
    }

    sw.Stop();

    Console.WriteLine($"Run {i + 1} took {sw.ElapsedMilliseconds} ms");
}

Output:

Run 1 took 499 ms
Run 2 took 300 ms
Run 3 took 276 ms
Run 4 took 275 ms
Run 5 took 273 ms
Run 6 took 256 ms
Run 7 took 288 ms
Run 8 took 309 ms
Run 9 took 285 ms
Run 10 took 280 ms
Run 11 took 292 ms
Run 12 took 308 ms
Run 13 took 283 ms
Run 14 took 267 ms
Run 15 took 290 ms
Run 16 took 276 ms
Run 17 took 277 ms
Run 18 took 286 ms
Run 19 took 283 ms
Run 20 took 273 ms

Here also there is difference between first and second run, but it is not as drastic as Entity Framework.

I am using EF Core 3.1

As per @Larnu's comment, it may be because of query plan caching of SQL server, so I have tried to run slightly different query in each iteration like below

ADO.NET version

var sqlCommand = new SqlCommand($"select * from Departments where ID > {i}", sqlConnection);

Output:

Run 1 took 494 ms
Run 2 took 274 ms
Run 3 took 304 ms
Run 4 took 276 ms
Run 5 took 731 ms
Run 6 took 475 ms
Run 7 took 576 ms
Run 8 took 276 ms
Run 9 took 275 ms
Run 10 took 291 ms
Run 11 took 271 ms
Run 12 took 253 ms
Run 13 took 269 ms
Run 14 took 262 ms
Run 15 took 270 ms
Run 16 took 303 ms
Run 17 took 261 ms
Run 18 took 296 ms
Run 19 took 275 ms
Run 20 took 661 ms

Entity framework version:

var departments = _sampleContext.Departments.Where(x=>x.ID > i).AsNoTracking().ToList();

Output:

Run 1 took 2377 ms
Run 2 took 274 ms
Run 3 took 272 ms
Run 4 took 260 ms
Run 5 took 276 ms
Run 6 took 281 ms
Run 7 took 319 ms
Run 8 took 506 ms
Run 9 took 318 ms
Run 10 took 265 ms
Run 11 took 269 ms
Run 12 took 276 ms
Run 13 took 283 ms
Run 14 took 256 ms
Run 15 took 253 ms
Run 16 took 258 ms
Run 17 took 277 ms
Run 18 took 298 ms

I guess still the difference between first and second query is quite drastic, so is it just because of query plan cache of SQL Server or something else?

Answer 1

I think that this will answer your question: https://github.com/do.net/efcore/issues/9347

Long story short, model building for the context on the first run, takes time. The bigger the model the more time it'll take.

You can spot the following findings:

Total ms to first query 126078

Total ms to second query 17

Total ms to first save 121

Total ms to second save 10

The team knows that the model building in the first query takes time and they have mitigated it a bit but as the comment mentions:

We've improved the model building perf a bit, but your best bet would be waiting for Compiled Models #1906