Entity Framework 6，从具有许多指向其他表的链接的大表中获取过滤数据的最佳策略

Question

Which of these ways is better:

1. to perform 10000 small queries with filtration conditions to table linked with many other tables (we need data from linked tables too (doing.include() in LINQ expression)) or;
2. to get the data from the table and linked tables piece by piece without any conditions (in a loop, in 10 iterations => 10 queries to db), load them in a collection and then during every iteration perform all analytical work by LINQ with collection?

I think that second way will be faster but what about memory and how to solve this problem according to the rules of clean architecture?

Answer 1

Neither of these two ways of solving the problem is look adequate. If you suddenly got in the same situation like me, let`s exhale, drink tea, take a walk in the park.

Then try to rewrite the algorithm from the beginning. Almost certainly you can divide it onto the two parts:

At first, one query to db to filter out clients with whom you will work further. That`s can look like:

var outdatedClients = db.Clients.Where(client => client.Orders.Any(order => order.Date < DateTime.Now.AddDays(-client.Term));

Then with foreach (var client in outdatedClients){} you can work with outdated orders by using .include() if it necessary. In most situations you can even avoid this cycle and get necessary data with one more query to db.