实体框架/ LINQ-从大型嵌套实体数据集中返回数据传输对象

Question

I am using WebAPI and Entity Framework to build a REST API that points to a large MSSQL database (~200 tables). The database is very normalized, so retrieving values that are useful to the consumer of the API requires a lot of drilling down into the tables.

In order to return useful data to the consumer, I have taken the approach of building models (or DTOs) using a factory pattern. However, I noticed that even though all the data was being returned in a nice format, there were performance issues due to lazy loading being enabled. In short, I was querying too much data even while I was returning only the data I needed.

So I resorted to turning off lazy loading and have attempted to grab the data explicitly using Include methods:

var accessions = db.AccessionParties
  .Include(ap => ap.Accession.AccessionParties.Select(ap2 => ap2.Party))

  .Include(ap => ap.Accession.AccessionParties.Select(ap2 => ap2.AccessionPartyPurposes.Select  (app => app.PartyAccessionPurposeType)))

  .Include(ap => ap.Accession.AccessionAnimals.Select(x => x.AnimalInformationType))

  .Include(ap => ap.Accession.AccessionAnimals.Select(x => x.Specimens.Select(y => y.AccessionTestRequestSpecimens.Select(z => z.AccessionTestRequest.LabTestOffering.TestOffering))))

  .ToList()
  .Select(a => modelFactory.CreateAccessionModel(a.Accession));

Below is an example of the factory method I'm using to generate the model, which includes nested factory methods as well to shape my related data entities.

  public AccessionModel CreateAccessionModel(Accession accession)
    {
        return new AccessionModel()
        {
            AccessionKey = accession.AccessionKey,
            SubmittedDate = accession.SubmittedDate,
            Parties = accession.AccessionParties
                                .Select(accessionParty => new { accessionParty = accessionParty, accessionParty.Party })
                                .Select(accessionParty => CreatePartyModel(accessionParty.Party)),
            Animals = accession.AccessionAnimals.Select(accessionAnimal => CreateAccessionAnimalModel(accessionAnimal))
        };
    }

Are there any patterns or practices for handling the above situation? I've seen some examples of a method that allows you to pass in an array of include statements, but I cannot think of a way to handle this in an elegant, efficient, pragmatic way. Any input would be much appreciated.

Answer 1

I do this in one of my methods passing in an array of strings like so (it's ugly because I'm using reflection to get the property names but hopefully you get the idea)

private IQueryable<T> AddIncludes<T>(IDatabase db) // my Entity context class implements IDatabase
{
    var props = typeof(IDatabase).GetProperties(BindingFlags.Public|BindingFlags.Instance);
    IQueryable<T> ret = null;
    foreach (var prop in props)
    {
        if (prop.PropertyType.Name == "IDbSet`1" &&
        prop.PropertyType.GetGenericArguments()[0] == typeof(T))
        {
            ret = (IQueryable<T>)prop.GetValue(db, null);
            break;
        }
    }
    var includes = GetIncludes((DbContext)db, typeof(T)); // this returns an IEnumerable<string> of the includes
    foreach (string include in includes) // replace string with a lambda
    {
        ret = ret.Include(include); // this is where the includes are added
    }
    return ret;
}

private ICollection<T> LoadObjectGraph<T>(IDatabase db, Func<T, bool> filter)
{
    var queryableWithIncludesAdded = AddIncludes<T>(db);
    return queryableWithIncludesAdded.Where(filter).ToList();
} 

Instead of using strings, you could use the same technique to pass lambdas to build your query

Answer 2

You can use automapper to map between the entities and the DTOs and together with the Projection you can execute the query and load only the columns your DTO needs. Check http://automapper.org/ and https://github.com/AutoMapper/AutoMapper/wiki/Queryable-Extensions

Hope that helps.