[英]c# find start date and end date based on a list of dates?
I have a database table with over 200K+ records and a column containing a Date ( NOT NULL
).我有一个包含超过 20 万条记录的数据库表和一个包含日期的列(
NOT NULL
)。 I am struggling to do a GroupBy
Date since the database is massive the query takes soooo long to process (like 1 minute or so).我正在努力做一个
GroupBy
Date,因为数据库很大,查询需要很长时间来处理(比如 1 分钟左右)。
My Theory:我的理论:
GroupBy
on so the query will be done in a shorter set of records..GroupBy
,这样查询将在较短的记录集中完成。 Here is my Model that I have to get the list:这是我必须得到列表的 Model:
registration.Select(c => new RegistrationViewModel()
{
DateReference = c.DateReference,
MinuteWorked = c.MinuteWorked,
});
DateReference
is the database column that I have to work with... DateReference
是我必须使用的数据库列... I am not pretty sure how to cycle through my list getting the dates start and end without taking too long.我不太确定如何在不花太长时间的情况下循环浏览我的列表以获取日期的开始和结束。
Any idea on how to do that?关于如何做到这一点的任何想法?
EDIT:编辑:
var registrationList = await context.Registration
.Where(c => c.Status == StatusRegistration.Active) // getting all active registrations
.ToRegistrationViewModel() // this is simply a select method
.OrderBy(d => d.DateReference.Date) // this takes long
.ToListAsync();
The GroupBy:分组:
var grpList = registrationList.GroupBy(x => x.DateReference.Date).ToList();
var tempList = new List<List<RegistrationViewModel>>();
foreach (var item in grpList)
{
var selList = item.Select(c => new RegistrationViewModel()
{
RegistrationId = c.RegistrationId,
DateReference = c.DateReference,
MinuteWorked = c.MinuteWorked,
}).ToList();
tempList.Add(selList);
}
This is my SQL table:这是我的 SQL 表:
This is the ToRegistrationViewModel()
function:这是
ToRegistrationViewModel()
function:
return registration.Select(c => new RegistrationViewModel()
{
RegistrationId = c.RegistrationId,
PeopleId = c.PeopleId,
DateReference = c.DateReference,
DateChange = c.DateChange,
UserRef = c.UserRef,
CommissionId = c.CommissionId,
ActivityId = c.ActivityId,
MinuteWorked = c.MinuteWorked,
Activity = new ActivityViewModel()
{
Code = c.Activity.Code,
Description = c.Activity.Description,
},
Commission = new CommissionViewModel()
{
Code = c.Commission.Code,
Description = c.Commission.Description
},
People = new PeopleViewModel()
{
UserId = c.People.UserId,
Code = c.People.Code,
Name = c.People.Name,
Surname = c.People.Surname,
Active = c.People.Active
}
});
There are multiple potential problems here这里有多个潜在问题
Lack of indexes缺乏索引
Your query uses the Status and DateReference, and neither looks to have an index.您的查询使用 Status 和 DateReference,而且看起来都没有索引。 If there are only a few active statuses a index on that column might suffice, otherwise you need a index on the date to speedup sorting.
如果只有几个活动状态,则该列上的索引可能就足够了,否则您需要日期上的索引来加速排序。 You might also consider a composite index that includes both columns.
您还可以考虑包含两列的复合索引。 An appropriate index should solve the sorting issue.
适当的索引应该可以解决排序问题。
Materializing the query实现查询
ToListAsync
will trigger the execution of the sql query, making every subsequent operation run on the client. ToListAsync
将触发 sql 查询的执行,使每个后续操作都在客户端上运行。 I would also be highly suspicious of ToRegistrationViewModel
, I would try changing this to an anonymous type, and only convert to an actual type after the query has been materialized.我也会对
ToRegistrationViewModel
高度怀疑,我会尝试将其更改为匿名类型,并且仅在查询实现后才转换为实际类型。 Running things like sorting and grouping on the client is generally considered a bad idea, but you need to consider where the actual bottleneck is, optimizing the grouping will not help if the transfer of data takes most time.在客户端上运行诸如排序和分组之类的东西通常被认为是一个坏主意,但您需要考虑实际瓶颈在哪里,如果数据传输花费大部分时间,优化分组将无济于事。
Transferring data传输数据
Fetching a large number of rows will be slow, no matter what.无论如何,获取大量行都会很慢。 The goal is usually to do as much filtering in the database as possible so you do not need to fetch so many rows.
目标通常是在数据库中进行尽可能多的过滤,这样您就不需要获取这么多行。 If you have to fetch a large amount of records you might use Pagination , ie combine OrderBy with Skip and Take to fetch smaller chunks of data.
如果您必须获取大量记录,您可能会使用Pagination ,即将 OrderBy 与 Skip 和 Take 结合起来以获取较小的数据块。 This will not save time overall, but can allow for things like progress and showing data continuously.
这不会总体上节省时间,但可以允许诸如进度和连续显示数据之类的事情。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.