PLinq本质上比System.Threading.Tasks.Parallel.ForEach快

Question

Summary : I changed from System.Threading.Tasks.Parallel.ForEach and Concurrent Data structure to a simple plinq (Parallel Linq) query. The speed up was amazing .

So is plinq inherently faster than Parallel.ForEach? Or is it specific to the task.

// Original Code
// concurrent dictionary to store results
var resultDict = new ConcurrentDictionary<string, MyResultType>();

Parallel.ForEach(items, item =>
        {
            resultDict.TryAdd(item.Name, PerformWork(source));
        });


// new code

var results =
            items
            .AsParallel()
            .Select(item => new { item.Name, queryResult = PerformWork(item) })
            .ToDictionary(kv => kv.SourceName, kv => kv.queryResult);

Notes : Each task (PerformWork) now runs between 0 and 200 ms. It used to take longer before I optimized it. That's why I was using the Tasks.Parallel library in the fist place. So I went from 2 seconds total time to ~100-200 ms total time, performing roughly the same work, just with different methods. (Wow linq and plinq are awesome!)

Questions :

1. Is the speed up due to using plinq vs Parallel.ForEach?
2. Is it instead simply the removal of the concurrent data structure (ConcurrentDictionary)? (Because it doesn't need to synchronize threads).
3. Based on the answer from this related question

Whereas PLINQ is largely based on a functional style of programming with no side-effects, side-effects are precisely what the TPL is for. If you want to actually do work in parallel as opposed to just searching/selecting things in parallel, you use the TPL.

Can I assume that because my pattern is basically functional (giving inputs produce new outputs without mutation), that plinq is the correct technology to use?

I'm looking for validation that my assumptions are correct, or an indication that I'm missing something.

Answer 1

It's not possible to use these 2 code samples to do a definitive comparison between Parallel.ForEach and PLINQ. The code samples are simply too different.

The first item that jumps out at me is the first sample uses ConcurrentDictionary and the second uses Dictionary . These two types have very different uses and performance characteristics. In order to get an accurate comparison between the two technologies you need to be consistent here with the types.

Answer 2

Based on the limited information you've provided in your sample (I asked for more details in a comment on the OP), I'm guessing sure you're seeing differences due to the partitioning algorithm that is used. You should read up on Chunk Partitioning vs. Range Partitioning in this blog post where he discusses how they differ and for which types of work they might be best suited for. Highly recommend you read that blog article as well as this one which goes into a little more detail on those two types along with two other types of partitioning that can be used, though not applicable to your sample, as well as giving some visual aids to better understand the partitioning. Finally, here's yet another blog post that discusses work partitioning and how it can affect you when the default partitioning algorithm doesn't make sense for your particular workload. That post actually refers to a great program that helps you visualize the partitioners at work that's part of a set of parallel samples from the PFX team .