Entity Framework - Best practice to resolve performance issue with lazy loading

Question

I've taken over a project with Entity Framework. The code seems to be good layered and done with good structure. The problem is as I've seen many times with Entity Framework, that they used lazy loading alot. The problems don't apear until the db got some data and the sql queries just peaks.

The solution take a great use of keeping the repositories small and just get one level of data and as I worked with some of the biggest performance issues, the most common issue is often resolved by adding specific function to the repository that loads the nested entities and use some dynamic queries.

ie GetCustomerWithOrderData that includes orders, order rows etc.

Sometimes I have to merge two queries by first get Customer (with included relations) and then get Orders (with included) and map them together by linq.

The queries is far more complex then the examples and the lazy loading could be in business layer, controllers or views so there is wuite alot to trace to resolve.

But I feel that the code is quite large and I have a hard time finding the future problems. What I now need is a good way to track when there is lazy loading, and also be able to tell what objects that need to be loaded on a specific call.

The best would be if I could track a specific action call, and get what sql that executes, how many times, loadtime etc.

The solution is build with MVC 3 and EF4, is there any performance to gain by upgrading to newer EF?

Answer 1

At a previous job this became an issue as developers would abuse the framework and create n+1 issues all over the place -- only manifesting themselves as the tables grew large. Plus the lazy loading would create problems with json serialization as well if you didn't specify the serialization depth to stop at 1 -- even at that it was strange that sometimes related objects would be there, at other times they wouldn't (depending on depth).

In the end, we turned off lazy loading entirely and forced the developers to make the second database call to get the children entities they desired. Plus, the team was instructed to leave Sql Server Profiler on while developing to ensure nothing that would kill performance was being created.

There were definitely cons to this approach as well, such as the extra round trips to the db, the extra lines of code, and the general lack of understanding from new developers to the team why we'd do such a thing. Some argued that we could use Includes in our queries, but at some point our repositories were becoming bloated by doing so, and simplicity/readability are important too. In the end, the performance problems became non-existent, so the trade-off of killing lazy loading was worth it in my opinion.

This isn't a direct answer to the question, you asked, but maybe it'll give you some additional insight. In your situation, I'd watch Profiler to see where the worst abuse is taking place and then fix it one at a time until your performance is once again acceptable. You'll probably find that your fix is to do something similar, and eliminate lazy loading entirely.

I'd love to see other answers to this problem because this is an important topic, in my opinion.