Linq query taking too long to process results

Question

I have 3 tables

• ERPEntry
• ERPEntryType
• ERPApp

I am trying to get data from these 3 tables using the below queries:.. The table ERPApp is in different context than the other 2 tables, that is the reason i am using 2 queries

var res = (from s in ERPDB.ERPEntrys
         join t in ERPDB.ERPEntryTypes
         on s.EntryTypeID equals t.EntryTypeID
         where s.UserIDAdded == '250176'
         select new {s.EntryTypeID, s.DateAdded, t.EntryTypeName, s.AppID }).OrderByDescending(d => d.DateAdded).Take(10).ToArray();

var erpResult = (from a in APPDB.ERPApps.AsEnumerable()
         join b in res on a.AppId equals b.AppId
         select new ERPInfo
         {
          EntryId = b.EntryID,
          EntryType = b.EntryTypeName,
          ERPApp = a.ApplicationName,
          DateAdded = b.DateAdded
         }).ToList();

I got the desired result but the 2nd query is taking almost 4 minutes to process and return result into varibale erpResult... any help on how can i resolve this performace issue ?

Answer 1

First, the answer:

var res = (from s in ERPDB.ERPEntries
join t in ERPDB.ERPEntryTypes
on s.EntryTypeID equals t.EntryTypeID
where s.UserIdAdded == "250176"
select new { s.EntryID, s.EntryTypeID, s.DateAdded, t.EntryTypeName, s.AppId })
.OrderByDescending(d => d.DateAdded).Take(10).ToArray();

/* Which immediately executes:

exec sp_executesql N'SELECT TOP (10) [t0].[EntryID], [t0].[EntryTypeID], 
[t0].[DateAdded], [t1].[EntryTypeName], [t0].[AppId]
FROM [dbo].[ERPEntry] AS [t0]
INNER JOIN [dbo].[ERPEntryType] AS [t1] ON [t0].[EntryTypeID] = [t1]. 
[EntryTypeID]
WHERE [t0].[UserIdAdded] = @p0
ORDER BY [t0].[DateAdded] DESC',N'@p0 varchar(8000)',@p0='250176'

*/

// Get the distinct AppID values in res
// Executes in memory
var distinctApps = (from r in res select r.AppId).Distinct();

// Query APPDB with the distinct values
var matchingApps = (from a in APPDB.ERPApps where distinctApps
.Contains(a.AppId) select new { a.AppId, a.ApplicationName }).ToArray();

/* Which immediately executes this efficient query:

exec sp_executesql N'SELECT [t0].[AppId], [t0].[ApplicationName]
FROM [dbo].[ERPApp] AS [t0]
WHERE [t0].[AppId] IN (@p0, @p1, @p2, @p3)',N'@p0 bigint,@p1 bigint,@p2 
bigint,@p3 bigint',@p0=101,@p1=123,@p2=125,@p3=129

*/ 
var erpResultWithAppNames = (from a in matchingApps
join b in res on a.AppId equals b.AppId
select new
{
EntryId = b.EntryID,
EntryType = b.EntryTypeName,
ERPApp = a.ApplicationName,
DateAdded = b.DateAdded
}).ToList();

Extra notes:

As already mentioned in the comments, the call to .AsEnumerable() on the APPDB.ERPApps table causes the whole table to be loaded into memory as objects.

Facts:

1. You had to load 25.000 rows into memory (and convert them to objects),

2. You are not only loading 25.000 rows. You are loading 25.000 x 173 cells and create 25.000 objects with 173 fields each.

3. if you could just load the two fields you need (AppId and ApplicationName) instead of all the 173 fields (with whatever data they have) the performance would improve, but would still be inefficient considering what you are trying to achieve.

4. The current performance problem is in part due to transferring the whole table with all its 173 fields to your server, and in part to mapping all these rows to objects and fields to properties. The impact of each can be measured as further research.

Linq To SQL. When checked with SqlProfiler:

from a in APPDB.ERPApps.AsEnumerable() join b ...

// PROFILED QUERY:
SELECT [t0].[AppId], [t0].[ApplicationName], [t0].[Field1], [t0].[Field2],
[t0].[Field3], [t0].[Field4], [t0].[Field5], [t0].[Field6], [t0].[Field7],
[t0].[Field8], [t0].[Field9], ... ... ... ... [t0].[Field176], [t0].[Field173],
FROM [dbo].[ERPApp] AS [t0]

But:

var erpResult = (from a in APPDB.ERPApps
select new { AppId = a.AppId, ApplicationName = a.ApplicationName }
).ToArray();

// PROFILED QUERY:

SELECT [t0].[AppId], [t0].[ApplicationName]
FROM [dbo].[ERPApp] AS [t0]