How can I improve this queries speed?

Question

I have a table with a list of customers. One customer has 0, 1 or more contract(s).

I must retrieve all the enabled customer, set them in a DTO and add the current contract to this DTO (If there is one)

For the moment, it is very slow (more than 10min).

CODE

List<CustomerOverviewDto> result = new List<CustomerOverviewDto>();    
customers= context.Customers.Where(c => c.IsEnabled).ToList();
    foreach (Customer customer in customers)
    {
        CustomerOverviewDto customerDto = GetCustomer(customer);
        Framework.Contract contract =
            customer.Contracts.Where(c => c.ContractEndDate >= DateTime.Today && c.ContractStartDate <= DateTime.Today)
                .FirstOrDefault();
        if (contract != null)
        {
            SetContract(customerDto, contract);
        }
        result.add(customerDto);
    }

Answer 1

Use projection to only return the columns you work with by using "Select". If you have 36 columns this will give you better results.

customers= context.Customers.Where(c => c.IsEnabled).Select(cust => new Customer
{
    Id = cust .Id
}).ToList();

https://www.talksharp.com/entity-framework-projection-queries

After that check in the queryplan if you have table scans or index scans. Try to avoid them by setting apropriate indexes.

Answer 2

I think the the problem is the query that retrieves the contract inside the loop. It would be better to retrieve all the data with one query like this:

var date = DateTime.Today;
var query =
    from customer in context.Customers
    where customer => customer.IsEnabled
    select new 
    {
        customer,
        contract = customer.Contracts.FirstOrDefault(c => c.ContractEndDate >= date && c.ContractStartDate <= date)
    };
var result = new List<CustomerOverviewDto>();
foreach (var entry in query)
{
    CustomerOverviewDto customerDto = GetCustomer(entry.customer);
    if (entry.contract != null)
        SetContract(customerDto, entry.contract);
    result.add(customerDto);
}

Answer 3

Ok so first of all, when you use .ToList(), you are executing the query right there, and pulling back every row that IsEnabled into the memory for working on. You want to do more on the database side.

result = context.Customers.Where(c => c.IsEnabled); //be lazy

Secondly, the query is only going to perform well and be optimised by the execution engine properly if it has indexes to work with.

Add some indexes on the fields that you are performing comparisons on.

Think about this line of code for example

    customer.Contracts.Where(c => c.ContractEndDate >= DateTime.Today && 
c.ContractStartDate <= DateTime.Today).FirstOrDefault();

Is you don't have a foreign key from customers to contracts, and you have no indexes on ContractStartDate and ContractEndDate it is going to perform extremely poorly and is going to be run once for every customer that 'IsEnabled'

Answer 4

It seems you only want to do some thing when returns a value. So you can add this in you initial query, and include the contracts:

customers= context.Customers
                           .Include(c => c.Contracts)
                           .Where(c => c.IsEnabled
                                     && c.Contracts.Any(con => con.ContractEndDate >= DateTime.Today && con .ContractStartDate <= DateTime.Today))
                           .ToList();

 foreach (Customer customer in customers)  
 {
    CustomerOverviewDto customerDto = GetCustomer(customer);
    Framework.Contract contract =
    customer.Contracts.Where(c => c.ContractEndDate >= DateTime.Today && c.ContractStartDate <= DateTime.Today)
        .First();
    SetContract(customerDto, contract);
 }

Answer 5

As I have no idea of what your domain model structure looks like or why you are not using navigation properties to map the CURRENT contract to the customer, you could do something like this.

You could do just 2 roundtrips to the database by materializing all the customers and contracts and then mapping them in memory to your DTO objects. Assuming you have CustomerId as FK and Customer.Id as PK.

List<CustomerOverviewDto> result = new List<CustomerOverviewDto>();    

customers = context.Customers.Where(c => c.IsEnabled).ToList();
contracts = context.Contracts.Where(c => c.ContractEndDate >= DateTime.Today && c.ContractStartDate <= DateTime.Today).ToList();

foreach (Customer customer in customers)
{
    var customerDto = GetCustomer(customer);
    var contract = contracts.Where(c => c.CustomerId == customer.Id).FirstOrDefault();
    if (contract != null)
    {
        SetContract(customerDto, contract);
    }

    result.add(customerDto);
}

Answer 6

I finally solved the problem by using 1 query and projection

context.Customers.Where(c => c.IsEnabled).Select(c => new CustomerOverviewDto{...}).ToList();

I directly retrieve the contract when creating the CustomerOverviewDto