Slow LINQ query against strongly typed Dataset

Question

I have a database with about 5,000 rows in it. There are also a number of many-to-many relationships. I need to do a free text search across tables as part of an "advanced search" query.

I've created a strongly-typed Dataset and import all the data from SQL Server when the app starts. When doing a LINQ query against the Dataset, the query executes very slowly (around 15 seconds). I thought executing the query against an in-memory Dataset would be much faster than SQL Server, but that doesn't seem to be the case. I even need to add more joins and "searches" in the where clause, so things are only going to get worse.

In the fields that I'm searching, the longest is Summary, and the longest one in the database is less than 2,000 bytes, so we're not talking about a lot of data to search through. Am I barking up the wrong tree here, or is there a way to improve the performance of this query?

Here's the code:

var results = from e in _data.ds.Employee
      join es in _data.ds.EmployeeSkill on e.EmployeeId equals es.EmployeeId into esGroup from esItem in esGroup.DefaultIfEmpty()
      join s in _data.ds.Skill on esItem?.SkillId equals s.SkillId into sGroup from skillItem in sGroup.DefaultIfEmpty()
      join er in _data.ds.EmployeeRole on e.EmployeeId equals er.EmployeeId into erGroup from erItem in erGroup.DefaultIfEmpty()
      join r in _data.ds.Role on erItem?.RoleId equals r.RoleId into rGroup from rItem in rGroup.DefaultIfEmpty()
      join et in _data.ds.EmployeeTechnology on e.EmployeeId equals et.EmployeeId into etGroup from etItem in etGroup.DefaultIfEmpty()
      join t in _data.ds.Technology on etItem?.TechnologyId equals t.TechnologyId into tGroup from tItem in etGroup.DefaultIfEmpty()
      where
        e.FirstName.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0 ||
        e.LastName.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0 ||
        e.RMMarket.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0 ||
        !e.IsSummaryNull() && e.Summary.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0
      select new SearchResult
      {
          EmployeeId = e.EmployeeId,
          Name = e.FirstName + " " + e.LastName,
          Title = e.Title,
          ImageUrl = e.IsImageUrlNull() ? string.Empty : e.ImageUrl,
          Market = e.RMMarket,
          Group = e.Group,
          Summary = e.IsSummaryNull() ? string.Empty : e.Summary.Substring(1, e.Summary.Length < summaryLength ? e.Summary.Length - 1 : summaryLength),
          AdUserName = e.AdUserName
      };

Answer 1

Some thoughts:

First, you're searching strings. If there is a lot to search, consider maintaining a full-text index to speed it up.

Second, Put the where clause before the join clauses. Things that filter out data should be as high up in a LINQ statement as possible. It is currently joining in a bunch of data for every row, even where it won't be used if the where clause is false.

Answer 2

Assuming you still load into DataSet s instead of list of objects (not enough information to translate that part), here is what I suggest:

Prejoin the data to use as your search index:

var searchBase = (from e in _data.ds.Employee
             join es in _data.ds.EmployeeSkill on e.EmployeeId equals es.EmployeeId into esGroup
             from esItem in esGroup.DefaultIfEmpty()
             join s in _data.ds.Skill on esItem?.SkillId equals s.SkillId into sGroup
             from skillItem in sGroup.DefaultIfEmpty()
             join er in _data.ds.EmployeeRole on e.EmployeeId equals er.EmployeeId into erGroup
             from erItem in erGroup.DefaultIfEmpty()
             join r in _data.ds.Role on erItem?.RoleId equals r.RoleId into rGroup
             from rItem in rGroup.DefaultIfEmpty()
             join et in _data.ds.EmployeeTechnology on e.EmployeeId equals et.EmployeeId into etGroup
             from etItem in etGroup.DefaultIfEmpty()
             join t in _data.ds.Technology on etItem?.TechnologyId equals t.TechnologyId into tGroup
             from tItem in etGroup.DefaultIfEmpty()
             select new {
                e.FirstName, e.LastName, e.RMMarket, e.Summary,
                e.EmployeeID, e.Title, e.ImageUrl, e.Group, e.AdUserName
             }).ToList();

Run the search against the loaded and joined data:

var results = from e in searchBase
          where
                e.FirstName.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0 ||
                e.LastName.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0 ||
                e.RMMarket.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0 ||
                !e.IsSummaryNull() && e.Summary.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0
          select new SearchResult {
              EmployeeId = e.EmployeeId,
              Name = e.FirstName + " " + e.LastName,
              Title = e.Title,
              ImageUrl = e.IsImageUrlNull() ? string.Empty : e.ImageUrl,
              Market = e.RMMarket,
              Group = e.Group,
              Summary = e.IsSummaryNull() ? string.Empty : e.Summary.Substring(1, e.Summary.Length < summaryLength ? e.Summary.Length - 1 : summaryLength),
              AdUserName = e.AdUserName
          };

BTW, your sample code shows no reason to do the joins as none of the join range variables are using in conditions or the answer, and you are left joining each one anyway, so just leaving them out would be the fastest solution.