如何使用LINQ简化（加速）从包含超过一百万条记录的数据库中选择查询

Question

我的应用程序中有两个模型：提取客户名称的客户和提取购买信息的付款。 结果是，我获得了每个客户在时间间隔中的采购清单fromDate和toDate 。 但是所有这些过程花费了太多时间。 因为客户的数据库大约是1.500条记录，付款= 50万。 那么我如何加快这一过程呢？

public async Task<List<SomeModel>> SomeMethod(DateTime? fromDate, DateTime? toDate)
{
    var clients = await _db.Clients.ToListAsync();

    var totals = new List<SomeModel>();

    foreach (var client in clients)
    {
        var payment = await _db.Payments.Where(pay => pay.ClientId == client.Id).Where(
            p =>
                DateTime.Compare(p.TradeDate, (DateTime)fromDate) >= 0 &&
                DateTime.Compare(p.TradeDate, (DateTime)toDate) <= 0).ToListAsync();
        var totalsByCust = new SomeModel{ Username = client.Username };
        foreach (var item in payment)
        {
            totalByCust.Bcf += item.Bcf;
            totalByCust.Ecn += item.Ecn;
            totalByCust.Ecbt += item.Ecbt;
            totalByCust.OpenGl += item.OpenGl;
            totalByCust.JeyK += item.JeyK;

        }
        totals.Add(totalByCust);
    }
    return totals;
}

Answer 1

确保在“向客户付款”上具有导航属性。 或者，您可以将所有客户端加载到词典中，因为只有1500个客户端。 这是使用导航属性执行的操作：

public async Task<List<SomeModel>> SomeMethod(DateTime? fromDate, DateTime? toDate)
{
  return _db.Payments
    .Where(p =>
        DateTime.Compare(p.TradeDate, (DateTime)fromDate) >= 0 &&
        DateTime.Compare(p.TradeDate, (DateTime)toDate) <= 0))
    .GroupBy(p=>p.ClientId)
    .Select(g=>new SomeModel { 
      UserName = g.First().Client.UserName,
      Bcf = g.Sum(p=>p.Bcf),
      Ecn = g.Sum(p=>p.Ecn),
      Ecbt = g.Sum(p=>p.Ecbt),
      OpenGl = g.Sum(p=>p.OpenGl),
      JeyK = g.Sum(p=>p.JeyK)
    })
    .ToListAsync();
}

Answer 2

使用SQLCommand将此代码移动到传统的SQL语句。 Linq查询是逐行工作的，使用批处理查询时可以提高效率。

Answer 3

合并查询可能是最简单的尝试：

public async Task<List<SomeModel>> SomeMethod(DateTime? fromDate, DateTime? toDate) {
    var totals = new List<SomeModel>();

    var clientswithpayments = await _db.Clients.Select(client => new {
        client.Username,
        payments =  _db.Payments.Where(pay => pay.ClientId == client.Id).Where(p =>
            DateTime.Compare(p.TradeDate, (DateTime)fromDate) >= 0 &&
            DateTime.Compare(p.TradeDate, (DateTime)toDate) <= 0)
    }).ToListAsync();

    foreach (var client in clientswithpayments) {
        var totalsByCust = new SomeModel { Username = client.Username };
        foreach (var payment in client.payments) {
            totalByCust.Bcf += payment.Bcf;
            totalByCust.Ecn += payment.Ecn;
            totalByCust.Ecbt += payment.Ecbt;
            totalByCust.OpenGl += payment.OpenGl;
            totalByCust.JeyK += payment.JeyK;

        }
        totals.Add(totalByCust);
    }
    return totals;
}

在查询中处理所有内容都比较棘手，但可能是正确的答案：

public async Task<List<SomeModel>> SomeMethod2(DateTime? fromDate, DateTime? toDate) {
    var totals = _db.Clients.Select(client => new {
        client.Username,
        payments = _db.Payments.Where(pay => pay.ClientId == client.Id).Where(p =>
            DateTime.Compare(p.TradeDate, (DateTime)fromDate) >= 0 &&
            DateTime.Compare(p.TradeDate, (DateTime)toDate) <= 0)
    }).Select(cwp => new SomeModel {
        Username = cwp.Username,
        Bcf = cwp.payments.Sum(),
        Ecn = cwp.payments.Ecn.Sum(),
        Ecbt = cwp.payments.Ecbt.Sum(),
        OpenGl = cwp.payments.OpenGl.Sum(),
        JeyK = cwp.payments.JeyK.Sum()
    }

    return await totals.ToListAsync();
}

Answer 4

您应该使用存储过程而不是Linq查询。