Linq query taking too long

Question

I wonder if someone can help me with this. I have a LINQ query - no problems with it, but it takes far too long to return data

var result = Context.paf_wgs84.Where(c => c.Postcode.Contains(postcode)).Take(15);

Very simple, the idea as the user is typing, using AJAX, it returns a set of 15 possible matches.

The problem is, there are 1.6 million records

Running the following code in management studio takes around 3 seconds

SELECT   code
FROM     paf_wgs84
WHERE    (code LIKE '%EC1%')

where as running the following code takes less than a second

SELECT   TOP 15  code
FROM     paf_wgs84
WHERE    (code LIKE '%EC1%')

Is there a way of doing something similar in LINQ without using .take() ?

Answer 1

You can try something like this. This will only return one column.

var result = Context.paf_wgs84.Where(c => c.Postcode.Contains(postcode)).Select(x=>new {x.Postcode}).Take(15);

The generated Sql statement will look like this.

/*
-- Region Parameters
DECLARE @p0 VarChar(1000) = '%s%'
-- EndRegion
SELECT TOP (15) [t0].[code]
FROM [paf_wgs84] AS [t0]
WHERE [t0].[code] LIKE @p0
*/

Answer 2

the problem might be that your contains method in the statemnt it is not being mapped to a like statement in sql and you end up getting all the rows into sql and then doing a cotains search in your web tier instead of doing the same in your DB.

Use SqlMethods for the same.. somethig as follows:

SqlMethods.Like(c.Postcode, string.Format("%{0}%",postcode));

sometimes you can also use the string methods like: String.StartsWith or String.Ends with but in this you cant..

Also - LIKE clauses starting % are rarely a good idea - not least, it can't make effective use of any index. You might have better performance using "full text search"; but this isn't directly available via LINQ

hope that helps your problem.

Answer 3

The performance of these types of queries has been greatly improved in SQL Server 2012 using the new FETCH and OFFSET statements (although I haven't seen much benchmarking data to date...).

The trouble is EF doesnt utilize these new features yet.

A possible solution is to use SQL Server 2012 and write a sproc with FETCH/OFFSET and then target this using EF. Definitely not a short term solution but I wanted to point out this option.

Answer 4

I altered the statement to

var result = Context.paf_wgs84.Where(c => c.Postcode.StartsWith(postcode)).Take(10).Select(c => c.Postcode).ToArray();

I used a foreach look to add to a string array - I think this is where I was losing time!

Thanks for everyones help though.