Entity Framework Core 3 模拟 PARTITION BY

Question

I'm trying to retrieve the latest record for every group using EF Core 3 but every possible LINQ query I came up with ends up with the InvalidOperationException exception Processing of the LINQ expression '...' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core Processing of the LINQ expression '...' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core .

According to some answers that worked in EF Core 2.2 this query should did the trick but it didn't

from lfv in dbo.ListingFlagValues
group lfv by lfv.ListingId into groups
select groups.OrderByDescending(x => x.Timestamp).FirstOrDefault();

One more option I've tried was

db.ListingFlagValues.GroupBy(x => x.Listing)
  .Select(x => new { Group = x, MaxTimestamp = x.Max(y => y.Timestamp) })
  .SelectMany(x => x.Group.Select(y => new { y.ListingId, ValueId = y.NewFlagValueId, y.Timestamp, x.MaxTimestamp }))
  .Where(x => x.Timestamp == x.MaxTimestamp);

The behavior I'm trying to achieve is like in the following query

SELECT
    ListingId,
    NewFlagValueId AS ValueId
FROM
    (SELECT
        ListingFlagValues.ListingId,
        NewFlagValueId,
        [Timestamp],
        MAX([Timestamp]) OVER (PARTITION BY  ListingFlagValues.ListingId) AS MaxTimestamp
    FROM
        ListingFlagValues        
    WHERE 
        FlagId = 1) as FlagValues
WHERE [Timestamp] = [MaxTimestamp]

Answer 1

The key here is "This may indicate either a bug or a limitation in EF Core" in the exception message (read "may indicate" as "indicates "). While EF Core 3.0 has improved the query translation, it still doesn't support many query patterns, especially on the result of GroupBy . And since it also removed client evaluation, queries which worked in 2.x because of silent client evaluation now are simply failing.

From the other side EF Core 3.0 has improved translation of the last item in a group pattern by utilizing the ROW_NUMBER OVER (PARTITION BY SQL construct. However it doesn't work for GroupBy results, so you have to do the grouping manually by using Distinct query for keys and correlated subquery for values.

For instance, the following LINQ query

from listingId in db.ListingFlagValues.Select(x => x.ListingId).Distinct()
from lfv in db.ListingFlagValues
    .Where(x => x.ListingId == listingId)
    .OrderByDescending(e => e.Timestamp)
    .Take(1)
select lfv

translates and executes successfully using the following SQL

  SELECT [t1].[Id], [t1].[ListingId], [t1].[NewFlagValueId], [t1].[Timestamp]
  FROM (
      SELECT DISTINCT [l].[ListingId]
      FROM [ListingFlagValues] AS [l]
  ) AS [t]
  INNER JOIN (
      SELECT [t0].[Id], [t0].[ListingId], [t0].[NewFlagValueId], [t0].[Timestamp]
      FROM (
          SELECT [l0].[Id], [l0].[ListingId], [l0].[NewFlagValueId], [l0].[Timestamp], ROW_NUMBER() OVER(PARTITION BY [l0].[ListingId] ORDER BY [l0].[Timestamp] DESC) AS [row]
          FROM [ListingFlagValues] AS [l0]
      ) AS [t0]
      WHERE [t0].[row] <= 1
  ) AS [t1] ON [t].[ListingId] = [t1].[ListingId]