Using LINQ to Entities dynamic JOIN

Question

So say you have a table of Keywords. For simplicity sake lets just say it has 2 fields an Id integer, Keyword varchar(100). A query comes in for multiple keywords. For example a query for "quick brown fox". The requirement would be that we will take any records where the Id selected contains at least once occurrence of all three keywords. Moreover, it can be a partial match using the StartsWith. I can use the PredicateBuilder to build the multiple OR clauses that will ultimately be needed but to also filter these records, I would need to execute a JOIN on the same table for each keyword. I should note that the Id field is not unique and can have multiple entries.

The SQL Looks more or less like this or should

select k1.Id
  from Keywords k1
 inner join Keywords k2 on k1.Id = k2.Id
 inner join Keywords k3 on k2.Id = k3.Id
 where k1.Keyword like @k1
   and k2.Keyword like @k2
   and k3.Keyword like @k3

The LINQ that I have so far would be

var predicate = PredicateBuilder.False<Keyword>();
foreach (string term in searchTerms)
{
   string temp = term;
   predicate = predicate.Or(p => p.Keyword.StartsWith(temp));
}
var keys = Keywords.AsExpandable().Where(predicate).ToList();

This will produce SQL that more or less looks like:

SELECT 
[Extent1].[Id] AS [Id]
FROM  [dbo].[Keywords] AS [Extent1]
WHERE ([Extent1].[Keyword] LIKE @p__linq__1 ESCAPE '~') OR ([Extent1].[Keyword] LIKE @p__linq__2 ESCAPE '~') OR ([Extent1].[Keyword] LIKE @p__linq__3 ESCAPE '~')

In order to use this result I would then have to do a DistinctBy and then JOIN back to my results. This can produce a huge memory requirement and I am trying to find a solution that does most of what I want on the server.

Answer 1

To rephrase your query, you're looking for the ID values such that a group of all of the keywords for that ID contains all of the search terms that you're looking for.

Rather than trying to use Join to create a group of all of the keywords for an ID, you can use GroupBy , and then when you have the group of all keywords sharing an ID it's simple enough to filter out only the ones in which the group contains all of the search terms.

var query = keywords.GroupBy(keyword => keyword.Id)
    .Where(group => searchTerms.All(term => 
        group.Any(keyword => keyword.Keyword.StartsWith(term)))
    .Select(group => group.Key);

Answer 2

Ultimately I was able to get it to produce the SQL I was looking for. Even though the likelyhood that this issue is unique to me, I will post my solution. Perhaps it will be useful to someone.

LINQ

var term = searchTerms[0];
var keys = Keywords.Where (k => k.keyword.StartsWith(term));
for(var i=1; i < searchTerms.Length; i++)
{
    var searchTerm = searchTerms[i];
    keys = 
        from k1 in keys
        join k2 in Keywords on k1.Id equals k2.Id
        where k2.keyword.StartsWith(searchTerm)
        select k1;
}

PRODUCED SQL:

-- Region Parameters
DECLARE @p__linq__0 VarChar(1000) = 'the%'
DECLARE @p__linq__1 VarChar(1000) = 'brown%'
DECLARE @p__linq__2 VarChar(1000) = 'fox%'
-- EndRegion
SELECT 
    [Extent1].[Id] AS [Id]
    FROM    [dbo].[Keywords] AS [Extent1]
    INNER JOIN [dbo].[Keywords] AS [Extent2] ON [Extent1].[Id] = [Extent2].[Id]
    INNER JOIN [dbo].[Keywords] AS [Extent3] ON [Extent1].[Id] = [Extent3].[Id]
    WHERE ([Extent1].[keyword] LIKE @p__linq__0 ESCAPE '~') AND ([Extent2].[keyword] LIKE @p__linq__1 ESCAPE '~') AND ([Extent3].[keyword] LIKE @p__linq__2 ESCAPE '~')

Of course I would put some checking to make sure that I have more than 1 searchTerm before beginning my loop. But you get the picture.