Linq query takes too long when using Func & OrderByDescending

Question

Consider this code:

public List<Clients> GetFilteredClients(DateTime? FromDate = null, 
                                        DateTime? ToDate = null,    
                                        int? fromLocationType = null, 
                                        int? toLocationType = null)
{
    Func<Clients, bool> fromDateFilter = f => true;
    if (FromDate.HasValue)
    {
        fromDateFilter = z => z.Insert_Date.Value.Date >= FromDate.Value.Date;
    }

    Func<Clients, bool> toDateFilter = f => true;
    if (ToDate.HasValue)
    {
        toDateFilter = z => z.Insert_Date.Value.Date <= ToDate.Value.Date;
    }

    Func<Clients, bool> fromLocationTypeFilter = f => true;
    if (fromLocationType.HasValue)
    {
        fromOrgFilter = z => z.LocationTypeId >= fromLocationType.Value;
    }

    Func<Clients, bool> toLocationTypeFilter = f => true;
    if (toLocationType.HasValue)
    {
        toLocationTypeFilter = z => z.LocationTypeId <= toLocationType.Value;
    }

    var filtered = DB_Context.Clients
        .Where(fromDateFilter)
        .Where(toDateFilter)
        .Where(fromLocationTypeFilter)
        .Where(toLocationTypeFilter)
        .OrderByDescending(k => k.Id)
        .Take(1000)
        .ToList();
    return filtered;
}

I have something like 100K records in the DB, I need only the top 1000 that answer to the requirements of:

.Where(fromDateFilter)
.Where(toDateFilter)
.Where(fromLocationTypeFilter)
.Where(toLocationTypeFilter)

However the execution time still takes something like 10 seconds.

Any idea why?

Answer 1

You must use Expression<Func<...>> rather than Func<...> . When you use Func , only the enumerable methods can be used on the queryable, which in this case means you first download everything to memory, and then do the filtering. If you switch over to Expression<...> , the O/RM will do the filtering on the DB server, rather than in your application.

Also, there's better ways to do what you're doing. For example, you can build the conditions like so:

var query = DB_Context.Clients.AsQueryable();

if (FromDate.HasValue) query = query.Where(...);
if (ToDate.HasValue) query = query.Where(...);

...

return query.OrderByDescending(k => k.Id).Take(1000).ToList();

Of course, this means that whatever DB provider you're using must be able to support the kind of filtering you're trying to do - you'll need to consult the documentation.

Answer 2

You are using delegates instead LINQ expressions. That leads to processing a data by your application and not by SQL Server.

LINQ expressions look like lambda expressions thanks for the syntax, but they are not same thing. The compiler takes a decision what to create (delegates or LINQ expressions) depending on the situation.

If an object implements the IQueriable interface, then the compiler uses the Queryable class and generates LINQ expression trees, which later can be translated into a SQL query or other form by the specific IQueryProvider .

Otherwise, the compiler uses extensions from the Enumerable class, which create iterators over source collection (all records from the table in your case).

As an example. The code bellow will be compilled into LINQ expressions.

// Source code
IQueryable<Clients> source = null;
IQueryable<Clients> result = source.Where(c => c.LocationTypeId >= 1);

// Compiller generated code
IQueryable<Clients> source = null;

Expression parameterC = Expression.Parameter(typeof(Clients), "c");

IQueryable<Clients> result = Queryable.Where<Clients>(
    source,
    Expression.Lambda<Func<Clients, bool>>(
        Expression.LessThanOrEqual(
            Expression.Property(
                parameterC ,
                typeof(Clients).GetProperty("LocationTypeId").GetGetMethod()
                ),
            Expression.Constant(1, typeof(int))
            ),
    new ParameterExpression[]
        {
            parameterC 
        }
    );

And this code uses delegates:

// Source code
IQueryable<Clients> source = null;
Func<Clients, bool> filter = c => c.LocationTypeId >= 1;

IEnumerable<Clients> result = source.Where(filter );

// Compiller generated code
IQueryable<Clients> source = null;
Func<Clients, bool> filter = c => c.LocationTypeId >= 1;

IEnumerable<Clients> result = Enumerable.Where(source, filter);

So, to solve you problem use Expression<Func<Clients, bool>> instead of Func<Clients, bool> :

IQueryable<Clients> result = DB_Context.Clients;

if (someFilter.HasValue)
    result = result.Where(c => c.SomeProperty == someFilter.Value);

// other filters

return query
    .OrderByDescending(k => k.Id)
    .Take(1000)
    .ToList();