LINQ select with many ToList inside takes too much time

Question

I've below select with LINQ query that takes too much time If I do ToList inside Select on AsQueryable .

Approch 1

Takes approx 10s

This is simplest representation of my data with different ToList()

Below MYTABLE has one to many relationship with COLUMN (another table) which has Description field

using profiler query has UNION ALL for all ToList but query itself execute within 100ms, seems like it takes to parse by C#

  // Project into usable data and enumerate
            var enumerated = await this.Context.MYTABLE.AsQueryable().Where(some condition).Select(x => new
            {
                Study1 = x.COLUMN.Select(j => j.Description1).ToList(),
                Study2 = x.COLUMN.Select(j => j.Description2).ToList(),
                Study3 = x.COLUMN.Select(j => j.Description3).ToList(),
                Study4 = x.COLUMN.Select(j => j.Description4).ToList(),
                Study5 = x.COLUMN.Select(j => j.Description5).ToList()
            }).ToListAsync(cancellationToken);

            var results = enumerated.Select(o => new
            {
                Result1 = string.Join(", ", o.Study1),
                Result2 = string.Join(", ", o.Study2),
                Result3 = string.Join(", ", o.Study3),
                Result4 = string.Join(", ", o.Study4),
                Result5 = string.Join(", ", o.Study5),
            }).ToList();

            return results;

Approch 2

Takes approx 2s

using profiler query is very small compare to previous, around 1/5 lines

// Project into usable data and enumerate
        var enumerated = await this.Context.MYTABLE.AsQueryable().Where(some condition).Select(x => new
        {
            x.COLUMN
        }).ToListAsync(cancellationToken);

        var results = enumerated.Select(o => new
        {
            Result1 = string.Join(", ", o.COLUMN.Select(j => j.Description1).ToList()),
            Result2 = string.Join(", ", o.COLUMN.Select(j => j.Description2).ToList()),
            Result3 = string.Join(", ", o.COLUMN.Select(j => j.Description3).ToList()),
            Result4 = string.Join(", ", o.COLUMN.Select(j => j.Description4).ToList()),
            Result5 = string.Join(", ", o.COLUMN.Select(j => j.Description5).ToList()),
        }).ToList();

        return results;

Why Approch 2 takes less time and query is also simpler then 1?

Is there any suggestion to improve LINQ performance

Edit

I notice in Approch 2 it will call database multiple times for subsequent ToList() but still with all those calculation, Database retrieves within few milliseconds while C# take much more time.

Approch 3

This is also take around 2s .

I just did ToList first in Approch 1.

 // Project into usable data and enumerate
  var enumerated2 =  await this.Context.MYTABLE.AsQueryable().Where(some condition).ToListAsync(cancellationToken);

            var enumerated = enumerated2.Select(x => new
            {
                Study1 = x.COLUMN.Select(j => j.Description1).ToList(),
                Study2 = x.COLUMN.Select(j => j.Description2).ToList(),
                Study3 = x.COLUMN.Select(j => j.Description3).ToList(),
                Study4 = x.COLUMN.Select(j => j.Description4).ToList(),
                Study5 = x.COLUMN.Select(j => j.Description5).ToList()
            }).ToList()

            var results = enumerated.Select(o => new
            {
                Result1 = string.Join(", ", o.Study1),
                Result2 = string.Join(", ", o.Study2),
                Result3 = string.Join(", ", o.Study3),
                Result4 = string.Join(", ", o.Study4),
                Result5 = string.Join(", ", o.Study5),
            }).ToList();

            return results;

I don't understand why Approch 3 took less time then Approch 1 where only difference is performing ToList method.

Answer 1

1. Why do you need to call .ToList() twice? Remove the first one, that will help a bit.
2. Try using .Include("Tablename") . You can read about it here: https://medium.com/@lucavgobbi/improving-performance-with-linq-b478cd77d105

This is probably the so-called "n+1 problem" you are facing. You can read about it here: https://www.brentozar.com/archive/2018/07/common-entity-framework-problems-n-1/