基于线程数量与执行时间的关系的OpenMP并行编程

Question

I have the following code in Visual Studio 2010:

#pragma omp parallel for num_threads(1)
for(int y=0;y<col;y++)
bands[parametersnumberPredictionBands+1][x][y] = hyperspectral[x][y][z];

The time taken to execute the code when num_threads(1) is less than num_threads(3).

To my understanding when more threads are use in parallel the time taken should be reduced. Can anyone explain why?

I am a beginner so any help is very appreciated.

Answer 1

Creating and destroying threads takes some time. When the work done by each thread is trivial, the amount of time it takes to create and destroy threads outweighs the time it takes to do the work itself.

Instead try runnign something like

const int N = 100000;
int A[N];
#pragma omp parallel for
for (int i = 0; i < N; i++) {
    A[i] = i;
}

Here the time taken will be significantly lower with 3 threads as the amount of work done outweighs the work to create and destroy each thread.

Maybe to see this more clearly another example will help. Say we want to parallelize a loop that iterates 10 times, each iteration takes 1 second, and each additional thread created takes 0.5 seconds.

A single thread will finish the task in 10 seconds. If we create another thread the working time can be split in half, resulting in only 5 seconds of work. But it takes 0.5 seconds to create the extra thread so the total run time is 5.5 seconds. It is still faster than originally, but doubling the number of threads did not half our run time .

Lets say we want to run our loop really fast and we create 9 additional threads for a total of 10 threads. The working time can now be split among all 10 threads, resulting in 1 second of work. However, creating 9 threads costs 4.5 seconds. Our total run time is now 5.5 seconds which is the same as running with only 2 threads! The overhead of creating threads outweighs the amount of work to be done. Continuing to add threads at this point will only slow our program down.

Essentially there is a point for all programs with a fixed amount of work where more threads will not speed up the run time. Large workloads outweigh the cost of thread creating for a larger number of threads. Small workloads are almost immediately dwarfed by the thread creation overhead.

TL;DR

Parallelism is best suited for large problems. Parallelizing trivial problems will only make them slower.