Entity Framework - write an efficient query

Question

I wrote an application in C# and the way I write queries is inefficient.

For example, I have 3 entities:

Revision, Page, PageLine

Each revision contains list of pages and each page contains list of page lines.

To retrieve a specific revision, I retrieve all revisions from the DB and perform the filtering locally.

Right now everything is working fine, but assuming the DB will be filled with a lot of data, the queries will be slow and inefficient.

//ViewModel
private async void GetRevisionData(int revisionId) 
{
    List<Page> eepromPages = revisionEEPROMDataService.GetEEPROMPages(revisionId).Result;;
}

//DataService
public async Task<List<Page>> GetEEPROMPages(int eepromRevId)
{
    string[] includes = { "Pages", "Pages.PageLines" };
    IEnumerable<RevisionEEPROM> list = (IEnumerable<RevisionEEPROM>)await dataService.GetAll(includes);
    return list.Where(r => r.Id == eepromRevId).SelectMany(p => p.Pages).ToList();
}

//GenericDataService    
public async Task<IEnumerable<T>> GetAll(string[] includes = null)
{
    using (DeployToolDBContex contex = _contexFactory.CreateDbContext())
    {
        if (includes != null)
        {
            var query = contex.Set<T>().AsQueryable();

            foreach (var include in includes)
                query = query.Include(include);

            return query.ToList();
        }
        else
        {
            IEnumerable<T> entities = await contex.Set<T>().ToListAsync();
            return entities;
        }
    }
}

How can I make the query efficient and retrieve only the relevant data?

Answer 1

First off, I would recommend getting rid of the Generic data service. (or Repository pattern) It does absolutely nothing for you with EF in this case.

Next, to make efficient queries you should start by looking at what is the minimum viable product you need to send through to the view. Define a ViewModel that represents exactly what the view needs and nothing more. From there, have EF project the data domain (entities) down to the view model.

For example, if you want to list pages for a given EEPROM ID, and you only need 5 properties from the Page entity, define something like a PageSummaryViewModel. Something like "GetRevisionData" should be a Controller action rather than within a ViewModel where the Controller/Presenter has access to the EF DbContext that is scoped to serve the request, or to a Unit of Work instance encompassing the DbContext. (Whether a web application or a desktop application will govern options for managing the lifetime scope for the DbContext)

As a bare-bones simple example where a DbContext is registered /wa Per-Web-Request lifetime scope:

// Controller:
public Task<IEnumerable<PageSummaryViewModel>> GetPagesForEEPROMRevision(int eepromRevisionId)
{
    var pages = await _context.RevisionEEPROM
        .Where(r => r.Id == eepromRevisionId)
        .SelectMany(r => r.Pages.Select(p => new PageSummaryViewModel
        {
            PageId = p.Id,
            // other page fields the view uses.
            Lines = p.PageLines.Select(pl => new PageLineSummaryViewModel
            {
                PageLineId = pl.Id,
                // other line fields the view uses.
            }).ToList();
        }).ToListAsync();
    return pages;
}

If this is a desktop application then you would need to manage the lifetime scope of the DbContext, the simplest being a using block in the Presenter or similar pattern managing the actions loading the data to display.

The key considerations for writing efficient queries with EF is to reduce data down to view models or DTOs to reduce the amount of data coming over the wire and stored in memory, then leveraging deferred execution so that the database can do the majority of the heavy lifting to filter data, again reducing considerably what has to come over the wire.

Patterns like the Generic Repository only serve to complicate both of these two objectives. The argument for them is usually to try and abstract away the fact that you're relying on EF. (either to dumb it down for other developers, make it so EF could be swapped out later, or to try and maximize code reuse, etc.) However, this leaves the system running with what amounts to the lowest common denominator where individual needs cannot benefit from everything that EF can offer, leading to slower, more complex solutions.