How does sequential loop run faster than Parallel loop in C#?

Question

I tried a very minimal example:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Collections.Concurrent;
using System.Diagnostics;

namespace TPLExample {
    class Program {
        static void Main(string[] args) {
            int[] dataItems = new int[100];
            double[] resultItems = new double[100];

            for (int i = 0; i < dataItems.Length; ++i) {
                dataItems[i] = i;
            }

            Stopwatch stopwatch = new Stopwatch();

            stopwatch.Reset();
            stopwatch.Start();
            Parallel.For(0, dataItems.Length, (index) => {
                resultItems[index] = Math.Pow(dataItems[index], 2);
            });
            stopwatch.Stop();
            Console.WriteLine("TPL Time elapsed: {0}", stopwatch.Elapsed);

            stopwatch.Reset();
            stopwatch.Start();
            for (int i = 0; i < dataItems.Length; ++i) {
                resultItems[i] = Math.Pow(dataItems[i], 2);
            }
            stopwatch.Stop();
            Console.WriteLine("Sequential Time elapsed: {0}", stopwatch.Elapsed);

            WaitForEnterKey();
        }

        public static void WaitForEnterKey() {
            Console.WriteLine("Press enter to finish");
            Console.ReadLine();
        }

        public static void PrintMessage() {
            Console.WriteLine("Message printed");
        }
    }
}

The output was:

TPL Time elapsed: 00:00:00.0010670
Sequential Time elapsed: 00:00:00.0000178
Press enter to finish

The sequential loop is way faster than TPL! How is this possible? From my understanding, calculation within the Parallel.For will be executed in parallel, so must it be faster?

Answer 1

Simply put: For only iterating over a hundred items and performing a small mathematical operation, spawning new threads and waiting for them to complete produces more overhead than just running through the loop would.

From my understanding, calculation within the Parallel.For will be executed in parallel, so must it be faster?

As generally happens when people make sweeping statements about computer performance, there are far more variables at play here, and you can't really make that assumption. For example, inside your for loop, you are doing nothing more than Math.Pow , which the processor can perform very quickly. If this were an I/O intensive operation, requiring each thread to wait a long time, or even if it were a series of processor-intensive operations, you would get more out of Parallel processing (assuming you have a multi-threaded processor). But as it is, the overhead of creating and synchronizing these threads is far greater than any advantage that parallelism might give you.

Answer 2

Parallel loop processing is beneficial when the operation performed within the loop is relatively costly. All you're doing in your example is calculating an exponent, which is trivial. The overhead of multithreading is far outweighing the gains that you're getting in this case.

Answer 3

This code example is practical proof really nice answers above.

I've simulated intensive processor operation by simply blocking thread by Thead.Sleep.

The output was:

• Sequential Loop - 00:00:09.9995500
• Parallel Loop - 00:00:03.0347901

_

class Program
{
    static void Main(string[] args)
    {
        const int a = 10;

        Stopwatch sw = new Stopwatch();
        sw.Start();

        //for (long i = 0; i < a; i++)
        //{
        //    Thread.Sleep(1000);
        //}

        Parallel.For(0, a, i =>
        {
            Thread.Sleep(1000);
        });

        sw.Stop();

        Console.WriteLine(sw.Elapsed);

        Console.ReadLine();
    }
}

Answer 4

The overhead of parallelization is far greater than simply running Math.Pow 100 times sequentially. The others have said this.

More importantly, though, the memory access is trivial in the sequential version, but with the parallel version, the threads have to share memory (resultItems) and that kind of thing will really kill you even if you have a million items.

See page 44 of this excellent Microsoft whitepaper on parallel programming: http://www.microsoft.com/en-us/download/details.aspx?id=19222 . Here is an MSDN magazine article on the subject: http://msdn.microsoft.com/en-us/magazine/cc872851.aspx