为什么双重GroupBy + ToList花费的时间太长？

Question

I have these queries :

    var Data = (from ftr in db.TB_FTR
                      join mst in db.TB_MST on ftr.MST_ID equals mst.MST_ID
                      join trf in db.TB_TRF on mst.TRF_ID equals trf.ID
                      select new CityCountyType { City = ftr.CITY, County = ftr.COUNTY, Type = trf.TYPE }
                      ).OrderBy(i => i.City).ThenBy(i => i.County);

var Data2 =
    Data.GroupBy(i => new {i.City, i.County, i.Type})
        .Select(group => new {Name = group.Key, Count = group.Count()})
        .OrderBy(x => x.Name)
        .ThenByDescending(x => x.Count)
        .GroupBy(g => new {g.Name.City, g.Name.County})
        .Select(g => g.Select(g2 => 
            new {Name = new {g.Key.City, g.Key.County, g2.Name.Type}, g2.Count})).ToList();

I'm trying to get a list of list of objects whose counties and cities are the same. But the second query is taking too long to give the result. I waited about 30 minutes but there is no answer, however the list Data has about 5000 records. How can I change these queries so that I can get list of lists that I want? Thanks in advance.

For example this query returns such a list :

{ Name = {{ City = New York City, County  = Bronx, Type = Type A }}, Count = 4 }

{ Name = {{ City = New York City, County  = Bronx, Type = Type B }}, Count = 8 }

{ Name = {{ City = New York City, County  = Bronx, Type = Type C }}, Count = 24 }

{ Name = {{ City = New York City, County  = Manhattan, Type = Type B }}, Count = 43 }

{ Name = {{ City = New York City, County  = Manhattan, Type = Type C }}, Count = 58 }

{ Name = {{ City = Seattle, County  = King County, Type = Type D }}, Count = 43 }

{ Name = {{ City = Seattle, County  = King County, Type = Type A }}, Count = 67 }

{ Name = {{ City = Seattle, County  =    Snohomish County, Type = Type C }}, Count = 67 }

I want to make this list to several lists like this :

List 1:

{ Name = {{ City = New York City, County  = Bronx, Type = Type A }}, Count = 4 }

{ Name = {{ City = New York City, County  = Bronx, Type = Type B }}, Count = 8 }

{ Name = {{ City = New York City, County  = Bronx, Type = Type C }}, Count = 24 }

List 2 :

{ Name = {{ City = New York City, County  = Manhattan, Type = Type B }}, Count = 43 }

{ Name = {{ City = New York City, County  = Manhattan, Type = Type C }}, Count = 58 }

List 3 :

{ Name = {{ City = Seattle, County  = King County, Type = Type D }}, Count = 43 }

{ Name = {{ City = Seattle, County  = King County, Type = Type A }}, Count = 67 }

List 4:

{ Name = {{ City = Seattle, County  =  Snohomish County, Type = Type C }}, Count = 67 }

Answer 1

Possibility 1: Your database isn't indexed to support your query (where and join clauses).

To find out, get the generated sql and look at the execution plan. If the plan says nested loop join -> clustered index scan, you've found the problem.

Possibility 2: You've found the n+1 problem.

In Linq's GROUP BY , a group consists of group keys and group members. However in most SQL implementations, GROUP BY gives you the group keys and aggregates. In order to get the members of a group, a separate query is issued. If there are n groups, then n queries must be issued (the +1 is the original query).

To find out, get the generated sql. If there's a bunch of extra queries being issued and any of them say clustered index scan, you've found the problem.

Possibility 3: You're actually issuing n^2 (~5,000,000) queries.

Well, you grouped twice, so it might be a doubly nested loop. Look at the generated sql and find out.

---

The simplest fix for all of this is to pull the 5,000 records into memory before grouping. An easy way to do this is to call ToList before calling GroupBy .