How to do multiple join with group-by in LINQ

Question

I want to show the categories of a book in one row. I can do this in the LINQPad 6 application, but when I do this via C# I get an error and I don't understand my mistake.

Table     Books                 Categories                 Books_Categories
   
      BookId | Name        CategoryId |   Name        Id  | BookId | CategoryId
      --------------       ----------------------     -------------------------
       1     | BookA         1        | CategoryA      1  |   1    |    1
       2     | BookB         2        | CategoryB      2  |   1    |    2
                                                       3  |   2    |    2

First I tried the code in the LINQPad 6 application. The following code runs there fine:

from bc in Books_Categories
join b in Books on bc.BookId equals b.Id
join category in Categories on bc.CategoryId equals category.Id
group category by new {b.Id, b.Name} into g

select new {
    g.Key.Id,
    g.Key.Name,
    CategoryName = g.Select(x => x.Name)
}

The result after running the code:

BookA | List<String> (CategoryA  CategoryB)     
BookB | List<String> (CategoryA)

I've integrated the code into the application, but I get an error that I don't understand. When I run this application:

public List<BookDetail> GetBookDetail()
{
    using (EBookContext context = new EBookContext())
    {
        var result = (from bc in context.Books_Categories
                      join b in context.Books on bc.BookId equals b.Id
                      join c in context.Categories on bc.CategoryId equals c.Id
                      group c by new { b.Id, b.Name } into g
                      select new BookDetail
                      {
                          BookId = g.Key.Id,
                          BookName = g.Key.Name,
                          CategoryName = string.Join(",",g.Select(x => x.Name).ToList()) // CategoryName is a string variable.
                      });

        return result.ToList();
    }
}

I get the following error:

Select(x => x.Name)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

How can I solve this problem?

Answer 1

To understand the cause of your problem, you should be aware of the difference between an IEnumerable and an IQueryable .

IEnumerable

An object that implements IEnumerable<...> represents a sequence of similar objects. You can get the first item of the sequence, and as long as you've got an item, you can ask for the next one.

The IEnumerable holds everything to do this. Usually the IEnumerable is supposed to be executed by your local process, hence it can use every method that your local process can call.

To enumerate the sequence GetEnumerator is called and repeatedly MoveNext() / Current:

IEnumerable<Customer> customers = ...
using (IEnumerator<Customer> enumerator = customers.GetEnumerator()
{
    while (enumerator.MoveNext())
    {
         // There is still a Customer
         Customer customer = enumerator.Current;
         ProcessCustomer(customer);
    }
}

Usually you won't enumerate using these low level methods. The sequence is enumerated using methods like foreach, or using LINQ methods that don't return IEnumerable<...> like ToList, TDictionary, FirstOrDefault, Count, Any, etc.

IQueryable

Although an IQueryable looks very similar to an IEnumerable, it does not represent a sequence, it represents the potential to get an enumerable sequence.

For this, the IQueryable holds an Expression and a Provider . The Expression holds in a generic format what data must be fetched. The Provider knows who must provide the data (usually a database management system), and the language that is used to communicate with the DBMS (usually SQL).

When you call a method that will enumerate the IQueryable, then deep inside GetEnumerator() / MoveNext() / Current are called. The Expression is sent to the Provider who will translate it into SQL and fetch the data from the DBMS. The returned data is represented as an IEnumerator<...> , of which you can call MoveNext() / Current.

As said above, as soon as you call GetEnumeator / MoveNext, the Expression is sent to the Provider, which will try to translate the data into SQL.

Alas, the Provider does not know your local functions, and thus can't translate them into SQL. And although the developers of entity framework did a smart job, several .NET methods also can't be translated into SQL. In fact there are even several LINQ methods that are not supported. See Supported and unsupported LINQ methods (LINQ to entities) .

What has this to do with my question?

In your Select you use String.Join . Your Provider does not know how to translate this into SQL. The compiler does not know how smart your Provider is, so the compiler can't complain. You'll see the problem at run time.

What should I do?

Consider to use AsEnumerable . This will move the selected data as an Enumerable sequence to your local process, where you can use String.Join.

However, be careful when using AsEnumerable. Database management systems are extremely optimized in searching and combining tables. The transport of the selected data to your local process is one of the slower parts of your query. So rewrite your query such that you don't transport (much) more data than you actually will use.

Therefore try to avoid using AsEnumerable before a Where , especially if the Where filters out most of the fetched items.

In your query it seems not a big problem to fetch all Categories of each BookDetail, and locally Join them: a database String.Join wouldn't have limited the number of bytes that would have been transported.

To get all Books, each with its Categories I use one of the overloads of Queryable.GroupJoin

var booksWithCategories = dbContext.Books.GroupJoin(dbContext.BooksCategories,

    book => book.Id,                      // from every Book take the Id
    bookCategory => bookCategory.BookId,  // from every BookCategory take the foreign key

    // Parameter resultSelector: from every Book, and all matching BookCategories
    // make one new:
    (book, bookCategories) => new
    {
        Id = book.Id,
        Name = book.Name,
        Categories = dbContext.Categories
            .Where(category => bookCategories
                               .Select(bookcategory => bookcategory.CategoryId)
                               .Contains(category.Id))
            .Select(category => new
            {
                // select the category properties that you want, for example:
                Id = category.Id,
                Name = category.Name,
            }
            .ToList(),
    })

In words: from every Book in the sequence of DbContext.Books, get the Id. From every bookcategory in the sequence of DbContext.BookCategories, get the BookId. When they Match, use the Book and all its matching BookCategories to make one new object. Take the Id and the Name from the Book.

To get the categories of the book, extract the CategoryId from all BookCategories that belonged to this Book. Result: a sequence of CategoryIds that belong to this Book.

Now get all Categories in DbContext.Categories and keep only those that have an Id that is in this sequence of CategoryIds. Use these Categories to select the Category properties that you want.

In your specific example, you only want the Name, so you can change property Categories into:

CategorieNames = dbContext.Categories
            .Where(...)                                       // same as above
            .Select(category => caegory.Name).ToList(),

But I want to join these strings!

As long as this data is used internally, I would not advice to make one string for it. This will only make it more difficult to reuse the fetched data.

Therefore I would advice to keep it as a list of Category Names as long as possible, and only Join it just before you decide to display it:

Continuing the query

.AsEnumerable()
.Select(book => new
{
    ... // Book properties
    CategoryNames = String.Join(", ", book.Categories);
}

Answer 2

Try .AsEnumerable() :

var result = (from bc in context.Books_Categories.AsEnumerable()
             join b in context.Books on bc.BookId equals b.Id
             join c in context.Categories on bc.CategoryId equals c.Id
             group c by new { b.Id, b.Name } into g
             select new BookDetail
             {
                 BookId = g.Key.Id,
                 BookName = g.Key.Name,
                 CategoryName = string.Join(",",g.Select(x => x.Name).ToList()) // CategoryName is a string variable.
             });