Performance issue loading data from database using EF5 and looping using foreach

Question

I am building a statistics (multi-agents) application, I'm using EF5 to retrive a list of agents from agents table. it's a very large amount of data (arround 50 000 agents per region) needed to be loaded at a time to proceed some immigration logic on them. I found that the (Region).Agents navigation property is very very slow to load, but, executing the same query takes just 1 sec on Sql server. and the foreach too takes a considerable time looping.

foreach (OrigineAgent origineAgent in new List<Agent> Origine.Agents.AsParallel()))
{
    origineAgent.ScoreCalculator = _container.Resolve<IScoreCalculator>();
    origineAgent.AgentExoVariables = AgentVars;
    _migrationManager.Migrate(origineAgent);
}

What do I have to do to increase data loading and looping performance?

Answer 1

How about that:

var resolved = _container.Resolve<IScoreCalculator>();
foreach (OrigineAgent origineAgent in Origine.Agents.ToList().AsParallel())
{
    origineAgent.ScoreCalculator = resolved
    origineAgent.AgentExoVariables = AgentVars;
    _migrationManager.Migrate(origineAgent);
}

I've moved _container.Resolve<IScoreCalculator>(); before the loop and fixed your foreach loop source, because the one from your question does not even compile.

Answer 2

Building on MarcinJuraszek's post above, how about following:

List<Task> tasks = new List<Task>();
var resolved = _container.Resolve<IScoreCalculator>();
foreach (OrigineAgent origineAgent in Origine.Agents.ToList().AsParallel())
{
    origineAgent.ScoreCalculator = resolved
    origineAgent.AgentExoVariables = AgentVars;
    Task t = Task.Factory.StartNew((arg) => { _migrationManager.Migrate((OrigineAgent)arg); }, origineAgent);
    tasks.Add(t);
}
Task.WaitAll(tasks.ToArray());

It parallelises calls to _migrationManager.Migrate(origineAgent);