in Parallel.ForEach : The underlying provider failed on Open. with EF5

Question

This code would work sometimes for some items but it always fails when trying to process more items and I would get this : {"The underlying provider failed on Open."} exception

  List<Recon> scenarioAll = db.Transactions
                  .Where(t => t.SrcObjTyp == "13")
                  .Select(t => t.Recon).ToList();


  //db.Transactions.Where(t => t.SrcObjTyp == "13").ToList().ForEach(t => reconsWithType13Trans.Add(t.Recon));

  Parallel.ForEach(scenarioAll.Take(100), r =>
  {

    // ### Exception : {"The underlying provider failed on Open."} here ###
    invoices = r.Transactions.SelectMany(t => t.InvoiceDetails).ToList();


    CreateFacts(invoices, r/*, db*/).ForEach(f => facts.Add(f));
    transactions = r.Transactions.Where(t => !t.SrcObjTyp.Contains("13")).ToList();
    DistributeTransactionOnItemCode(transactions, facts);
    Console.WriteLine(i += 1);
  });

  facts.ForEach(f => db.ReconFacts.Add(f));
  db.SaveChanges();

I think the issus is multiple process accessing the database at the same time is it possible to allow EF to be queried by multiple process this way ?

Also is there a way to load everything in memory so when accessing the child of Recon like that r.Transactions.SelectMany(t => t.InvoiceDetails).ToList(); everything would be in memory and not accessing the underlying database ?

What solution should I use what do you think is best ?

Answer 1

Your issue is that you have multiple threads trying to Lazy load. Try replacing your load query with...

List<Recon> scenarioAll = db.Transactions
              .Where(t => t.SrcObjTyp == "13")
              .Select(t => t.Recon)
              .Include(r => r.Transactions.Select(t => t.InvoiceDetails))
              .ToList();

http://msdn.microsoft.com/en-us/library/gg671236(v=vs.103).aspx

Generally speaking if you are relying on Lazy Loading you are doing it wrong.

Alternatively, given that you might not need Transactions...you could do this...

 var query =  = db.Transactions
              .Where(t => t.SrcObjTyp == "13")
              .Select(t => t.Recon);
 var scenarioAll = query.ToList();
 var invoicesByReconIdQuery = from r in query
                              from t in r.Transactions
                              from i in t.InvoiceDetails
                              group i by r.Id into g
                              select new { Id = g.Key, Invoices = g.ToList() };


 var invoicesByReconId = invoicesByReconIdQuery.ToDictionary(x => x.Id, x => x.Invoices);

 Parallel.ForEach(scenarioAll.Take(100), r =>
 {

     // ### Exception : {"The underlying provider failed on Open."} here ###
     var invoices = invoicesByReconId[r.Id];
 }