How can i speed up these linq queries?

Question

Can i combine the following two linq queries in an single one, for speeding things up?

The first one, searches and performs the paging

 Products.Data = db.Products.Where(x =>  x.ProductCode.Contains(search) ||
                                         x.Name.Contains(search) || 
                                         x.Description.Contains(search) || 
                                         x.DescriptionExtra.Contains(search) ||
                                         SqlFunctions.StringConvert(x.Price).Contains(search) ||
                                         SqlFunctions.StringConvert(x.PriceOffer).Contains(search) ||
                                         SqlFunctions.StringConvert(x.FinalPrice).Contains(search) ||
                                         SqlFunctions.StringConvert(x.FinalPriceOffer).Contains(search))
                                         .OrderBy(p => p.ProductID)
                                         .Skip(PageSize * (page - 1))
                                         .Take(PageSize).ToList();

while the second one counts the total filtered results.

    int count = db.Products.Where(x => x.ProductCode.Contains(search) ||
                                       x.Name.Contains(search) ||
                                       x.Description.Contains(search) ||
                                       x.DescriptionExtra.Contains(search) ||
                                       SqlFunctions.StringConvert(x.Price).Contains(search) ||
                                       SqlFunctions.StringConvert(x.PriceOffer).Contains(search) ||
                                       SqlFunctions.StringConvert(x.FinalPrice).Contains(search) ||
                                       SqlFunctions.StringConvert(x.FinalPriceOffer).Contains(search))
                                      .Count();

Answer 1

Get rid of your ridiculously inefficient conversions.

SqlFunctions.StringConvert(x.Price).Contains(search) ||

No index use possible, full table scan, plus a conversion - that is as bad as it gets.

And make sure you have all indices.

Nothing else you can do.

Answer 2

Stop using 'contains' function cause it's a very slow thing (if you can)

Make sure that Your queries can make use of index`es in the DB. If You MUST have 'contains' - the take a look at full-text search capabilieties of SQL, but You might need to change this so pure sql or customise how your Linq is translated to SQL to make use of Full-Text indexes

Answer 3

I do not think You can combine them directly. This is one problem of paging - You need to know the total count of result anyway. Problem of dynamic paging further is, that one page can be inconsistent with another, because it is from different time. You can easily miss item completely because of this. If this can be a problem, I would avoid dynamic paging. You can fill ids of the whole result into some temporary table on server and do paging from there. Or You can return all ids from fulltext search and query the rest of data on demand.

There are some more optimizations, You can start returning results when the search string is at least 3 characters long, or You can build special table with count estimates for this purpose. You can also decide, that You would return only first ten pages and save server storage for ids (or client bandwidth for ids).

I am sad to see "Stop using contains" answers without alternative. Searching in the middle of the words is many times a must. The fact is, SQL server is terribly slow on text processing and searching is no exception. AFAIK even full-text indexes would not help You much with in-the-middle substring searching.

For the presented query on 10k records I would expect about 40ms per query to get counts or all results (my desktop). You can make computed persisted colum on this table with all texts concatenated and all numbers converted and query only that column. It would speed things up significantly (under 10ms for query on my desktop).

[computedCol]  AS (((((((((([text1]+' ')+[text2])+' ')+[text3])+' ')+CONVERT([nvarchar](max),[d1]))+' ')+CONVERT([nvarchar](max),[d2]))+' ')+CONVERT([nvarchar](max),[d3])) PERSISTED