Increase EF query performance

Question

I currently have the following code in my project

   public List<PermissionValue> GetUnderlyingPermissionsForUser(string userName, int guiPermissionTypeId, int productTypeId)
    {
        using (CliVeEntities db = new CliVeEntities())
        {             
            var listUnderlyings = (from gui in db.GuiPermissionUnderlying
                                join bue in db.BranchUsersExternal on gui.BranchUsersId equals bue.BranchUsersID     
                                join bu in db.BranchUsers on bue.BranchUsersID equals bu.BranchUsersId
                                where bu.UserName == userName && gui.ViewTypeId == guiPermissionTypeId
                                && gui.UnderlyingId.HasValue
                                && !gui.SectorId.HasValue
                                && gui.ProductTypeId == null
                                && gui.ProductGroupId == null
                                select new PermissionValue { Id = gui.UnderlyingId.Value, Visible = gui.value}).ToList<PermissionValue>();


            var listUnderlyingsProductGroup = (from gui in db.GuiPermissionUnderlying
                                                join bue in db.BranchUsersExternal on gui.BranchUsersId equals bue.BranchUsersID
                                               join bu in db.BranchUsers on bue.BranchUsersID equals bu.BranchUsersId
                                               join pg in db.ProductGroup on gui.ProductGroupId equals pg.ProductGroupId
                                               join pt in db.ProductType on pg.ProductGroupId equals pt.ProductGroupId
                                               where bu.UserName == userName && gui.ViewTypeId == guiPermissionTypeId
                                               && gui.UnderlyingId.HasValue
                                               && !gui.SectorId.HasValue
                                               && pt.ProductTypeId == productTypeId
                                               select new PermissionValue { Id = gui.UnderlyingId.Value, Visible = gui.value }).ToList<PermissionValue>();




            var listUnderlyingsProductType = (from gui in db.GuiPermissionUnderlying
                                                join bue in db.BranchUsersExternal on gui.BranchUsersId equals bue.BranchUsersID
                                              join bu in db.BranchUsers on bue.BranchUsersID equals bu.BranchUsersId
                                              where bu.UserName == userName && gui.ViewTypeId == guiPermissionTypeId
                                              && gui.UnderlyingId.HasValue
                                              && !gui.SectorId.HasValue
                                              && gui.ProductTypeId == productTypeId
                                              select new PermissionValue { Id = gui.UnderlyingId.Value, Visible = gui.value }).ToList<PermissionValue>();


            var listUnderlyingsForSectors = (from gui in db.GuiPermissionUnderlying
                                             join bue in db.BranchUsersExternal on gui.BranchUsersId equals bue.BranchUsersID
                                             join bu in db.BranchUsers on bue.BranchUsersID equals bu.BranchUsersId
                                             join u in db.Underlying on gui.SectorId equals u.SectorId
                                             where bu.UserName == userName && gui.ViewTypeId == guiPermissionTypeId
                                             && !gui.UnderlyingId.HasValue
                                             && gui.ProductTypeId == null
                                             && gui.ProductGroupId == null
                                             select new PermissionValue { Id = u.UnderlyingId, Visible = gui.value }).ToList<PermissionValue>();


            var listUnderlyingsForSectorsProductGroup = (from gui in db.GuiPermissionUnderlying
                                                        join bue in db.BranchUsersExternal on gui.BranchUsersId equals bue.BranchUsersID
                                                         join bu in db.BranchUsers on bue.BranchUsersID equals bu.BranchUsersId
                                                         join u in db.Underlying on gui.SectorId equals u.SectorId
                                                         join pg in db.ProductGroup on gui.ProductGroupId equals pg.ProductGroupId
                                                         join pt in db.ProductType on pg.ProductGroupId equals pt.ProductGroupId
                                                         where bu.UserName == userName && gui.ViewTypeId == guiPermissionTypeId
                                                         && !gui.UnderlyingId.HasValue
                                                         && pt.ProductTypeId == productTypeId
                                                         select new PermissionValue { Id = u.UnderlyingId, Visible = gui.value }).ToList<PermissionValue>();

            var listUnderlyingsForSectorsProductType = (
                                                        from gui in db.GuiPermissionUnderlying
                                                        join bue in db.BranchUsersExternal on gui.BranchUsersId equals bue.BranchUsersID
                                                        join bu in db.BranchUsers on bue.BranchUsersID equals bu.BranchUsersId
                                                        where bu.UserName == userName && gui.ViewTypeId == guiPermissionTypeId
                                                        && !gui.UnderlyingId.HasValue
                                                        && gui.SectorId.HasValue
                                                        && gui.ProductTypeId == productTypeId
                                                        select new PermissionValue { Id = gui.UnderlyingId.Value, Visible = gui.value }).ToList<PermissionValue>();

            var mergeList1 = MergeAndDistinctList(listUnderlyings, listUnderlyingsProductType);
            var mergeList2 = MergeAndDistinctList(mergeList1, listUnderlyingsProductGroup);
            var mergeList3 = MergeAndDistinctList(mergeList2, listUnderlyingsForSectors);
            var mergeList4 = MergeAndDistinctList(mergeList3, listUnderlyingsForSectorsProductType);
            var resultMergeList = MergeAndDistinctList(mergeList4, listUnderlyingsForSectorsProductGroup);

            return resultMergeList;

And This is the code for my MergeAndDistinctList function

 protected List<PermissionValue> MergeAndDistinctList(List<PermissionValue> listPrimary, List<PermissionValue> listSecondry)
    {
        List<PermissionValue> listMergedAndDistinct = new List<PermissionValue>();

        listMergedAndDistinct.AddRange(listPrimary);

        var filter = listPrimary.Select<PermissionValue, int>(p => p.Id);
        listMergedAndDistinct.AddRange(listSecondry.Where<PermissionValue>(p => !filter.Contains(p.Id)).Select(p => p));
        return listMergedAndDistinct;
    }

My problem with my GetUnderlyingsForClient() code is that it hits the database multiple times. If this method gets hit quite a lot then this could translate to multiple calls to the database.

Does anybody know of any ways to make this code more efficient and reduce the amount of DB calls i have to make.

Answer 1

It seems that you make a lot of queries on the same table (a in your case)

To limit database queries, you can get all data needed in a local list and use this list for your multiple queries like this :

var MyBaseList = db.A.ToList();

var listUnderlyings = (from a MyBaseList [...]
var listUnderlyingsProductGroup = (from a MyBaseList [...]

And for limit the data retrieval, filter MyBaseList to get only lines used by following queries.

Answer 2

You could Union your queries following this simplified pattern:

(from a in As where a.Id > 0 select a)
.Union(
from a in As where a.Name == "a" select a)
.Union(
...
)
.Select(a => new { a.x, a.y })
.Distinct()

The Union is the merge part of your MergeAndDistinctList and the Distinct , ehh, the distinct part. The nice thing is: the Distinct is carried out over the projection (Id and Visible in your case).

Everything should be carried out in one (monster) query now, without the intermediate forced execution that is caused by the AddRange in MergeAndDistinctList.

Looking at your queries I have a feeling that you could do with a smaller number of them by combining some conditions, but that's up to you.