OpenMP C ++：并行化for循环的负载不平衡

Question

I'm trying to parallelise a for loop, and I'm encountering undesired behaviour. The loop calls a taxing function (which contains another for loop) and then prints the result. I've parallelised the loop using #pragma omp parallel for .

The behaviour I'm seeing is: the CPU gets fully utilised at the start, and then near the end, it suddenly drops back down to 25% utilisation. My guess is that one task gets allocated to a thread, and then as most of the tasks get completed, the system waits for the newer ones to complete. Though if that were the case, I would've seen drops to 75%, 50%, and then 25%, but no, it drops straight to 25%.

I've tried to parallelise the function itself, but it made no difference. Removing the parallelisation on the loop resulted in a behaviour where usage would spike to 100%, then drop to 25%, and then repeat like that throughout execution, which resulted in even worse performance than before. I also tried a bunch of other options for the for loop like schedule.

How would I be able to assign unused threads to the last newly created tasks? Or is something like this not possible in OpenMP?

Answer 1

If your guess is correct, then you should apply schedule(dynamic) to your loop, which has the following effect:

When kind is dynamic , the iterations are distributed to threads in the team in chunks. Each thread executes a chunk of iterations, then requests another chunk, until no chunks remain to be distributed. Each chunk contains chunk_size iterations, except for the chunk that contains the sequentially last iteration, which may have fewer iterations. When no chunk_size is specified, it defaults to 1.

You can also experiment with increasing the chunk_size (eg, schedule(dynamic,16) ) or using schedule(guided) :

When kind is guided , the iterations are assigned to threads in the team in chunks. Each thread executes a chunk of iterations, then requests another chunk, until no chunks remain to be assigned. For a chunk_size of 1, the size of each chunk is proportional to the number of unassigned iterations divided by the number of threads in the team, decreasing to 1. [...]

Take a look at this answer for a detailed discussion about dynamic vs guided schdules.

In general, I recommend to never guess about performance . Use a sophisticated performance analysis tool that understands OpenMP and can tell you about the actual potential for optimization in your code.