Dynamic query execution in Entity Framework Core

Question

I am developing a hospitality domain application (.Net Core 2.2) in which i am developing a reporting module. From a dashboard few filters are available to fetch records from Data Base.

Below is the DTO i am using to contains filters

public class SearchDto
{
    public DateTime DateForm {get;set;}
    public DateTime DateTo {get;set;}
    public DateSearchType SearchType {get;set;}
    public string RegionId {get;set;}
    public OrdersStatus status {get;set;}
    public string PaymentModeTypes {get;set;}
    public string channel {get;set;}
}

Here DateSearchType is a enum with value

1. Start // service Start Date
2. End // service End Date
3. Creation // Order Creation Date

Also OrdersStatus (an enum) with values like All, Confirmed, Cancelled, Pay.netFailed etc

PaymentModeTypes can be a single string or comma seprated string for ex: "NetBanking, CreditCard, DebitCard, Cash"

RegionId is also a single string or comma seprated string as "101, 102, 102"

Same for Channel either "Web" or "Web, Mobile"

Curently ef core expressssion i am using is as follow

var v = Database.Orders.List(
              x => ((SearchType == DateSearchType.Start) ? x.Services.Any(y => (y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date)) || DateForm.Date.Equals(DateTime.MinValue) : true)
              && ((SearchType == DateSearchType.End) ? x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date) : true)
              && ((SearchType == DateSearchType.Creation) ? x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date : true)
              && (RegionId.Length == 0 || RegionId.Contains(x.RegionId))
              && (status == OrdersStatus.All || x.Status == status)
              && (string.IsNullOrEmpty(PaymentModeTypes) || PaymentTypes.Contains(x.PaymentType))
              && (string.IsNullOrEmpty(channel) || channels.Contains(x.ChannelName)), //Where
                     x => x.Guests, 
                     x => x.Services 
                     );

Here Guests and Services are two another tables set as navigation property in Orders Model

This expression is working fine but taking too much time to execute, any good approach to optimize it or right way to rewrite this code?

Also what is the best practice to exclude any filter if its value is not provided.

Few Filters are not mandatory, they can be supplied or not, so query execution has to be of dynamic nature. current implementation if a filter value is not supplied

 && (string.IsNullOrEmpty(PaymentModeTypes) || PaymentTypes.Contains(x.PaymentType))

Can anybody suggest some good material or any piece of code regarding this so i can optimize it.

Please ignore Typo as i am not habitual to use dark theme

Edit 1:Client Evaluation set as off

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder
            .ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning));
    }

Answer 1

There is some inefficiency in the generated SQL which might be causing the performance issue.

Before EF Core one is expected to conditionally chain multiple Where calls or use some predicate builder utility to build conditionally Where predicate having only the necessary conditions.

This normally is not needed in EF Core, because it tries to automatically eliminate such conditions. It does that for logical expressions ( || , && ), but failing to do so for the conditional expressions ( ? : ). So the solution is to replace the later with the equivalent logical expressions.

You are doing that for most of your conditions, but not for the first 3. So replace

x => ((SearchType == DateSearchType.Start) ? x.Services.Any(y => (y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date)) || DateForm.Date.Equals(DateTime.MinValue) : true)
    && ((SearchType == DateSearchType.End) ? x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date) : true)
    && ((SearchType == DateSearchType.Creation) ? x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date : true)

with

x => (SearchType != DateSearchType.Start || x.Services.Any(y => y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date) || DateForm.Date.Equals(DateTime.MinValue))
    && (SearchType != DateSearchType.End || x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date))
    && (SearchType != DateSearchType.Creation || x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date)

and see if that helps. For sure the generated SQL will be optimal.

---

Noticed that you are using different variable names for comma separated strings and Contains filters. So I'm assuming you have something like this

public IEnumerable<string> PaymentTypes => (PaymentModeTypes ?? "").Split(", ");
public IEnumerable<string> channels => (channel ?? "").Split(", ");

which is good and will generate SQL IN (...) conditions when necessary.

You might consider doing the same for regions, eg add

public IEnumerable<string> RegionIds => (RegionId ?? "").Split(", ");

and replace

&& (RegionId.Length == 0 || RegionId.Contains(x.RegionId))

with

&& (string.IsNullOrEmpty(RegionId) || RegionIds.Contains(x.RegionId))