高效查询每个唯一ID的前N行

Question

This is a follow-up to this question.

TLDR:

The question:

I want to filter a query to only keep the first n rows for each unique ID.

The answer:

query = query.GroupBy(q => q.ID).SelectMany(g => g.Take(n));

The problem with this answer is that for 80,000+ rows, evaluating the query takes much longer than filtering by iteration ( foreach ) (at least twice as slow). Looking at the SQL generated by this answer, a CROSS APPLY is used, most likely for the SelectMany() .

This link describes what CROSS APPLY does:

The APPLY operator allows you to join two table expressions; the right table expression is processed every time for each row from the left table expression.

In short, I'm looking for a filtering query which efficiently gathers the top N rows for each unique ID .

A Linq solution with explained SQL would be ideal.

Answer 1

I found my answer in SQL here (SQL 2000 Solution at the bottom) and managed to implement a Queryable/Linq version:

query = tableQueryable.Where(a =>
          tableQueryable.Where(b => b.ID == a.ID)
            .OrderByDescending(o => o.Timestamp)
            .Take(N)
            .Select(s => s.PK)
          .Contains(a.PK)
        ).OrderByDescending(d => d.Timestamp);

A fairly standard "sub-query" pattern. It's much faster on a large table.

Answer 2

L2S does not have row number so Martin's trick cannot be used. I have been through this problem as well and as far as I ever found out this is the optimal L2S solution (that does not use native SQL in any way).

You can try pulling down all results into the application and doing the row number thing there. This can hurt or benefit performance. Which one it is depends on the concrete case.