OpenMP中的信令

Question

I am writing computational code that more-less has the following schematic:

#pragma omp parallel
{
    #pragma omp for nowait
    // Compute elements of some array A[i] in parallel

    #pragma omp single
    for (i = 0; i < N; ++i) {
        // Do some operation with A[i].
        // This time it is important that operations are sequential. e.g.:
        result = compute_new_result(result, A[i]);
    }
}

Both computing A[i] and compute_new_result are rather expensive. So my idea is to compute the array elements in parallel and if any of the threads gets free, it starts doing sequential operations. There is a good chance that the starting array elements are already computed and the others will be provided by the other threads doing still the first loop.

However, to make the concept work I have to achieve two things:

1. To make OpenMP split the loops in alternative way, ie for two threads: thread 1 computing A[0] , A[2] , A[4] and thread 2: A[1] , A[3] , A[5] , etc.

2. To provide some signaling system. I am thinking about an array of flags indicating that A[i] has already been computed. Then compute_new_result should wait for the flag for respective A[i] to be released before proceeding.

I would be glad for any hints how to achieve both goals. I need the solution to be portable across Linux, Windows and Mac. I am writing the whole code in C++11.

---

Edit:

I have figured out the answer to the fist question. It looks like it is sufficient do add schedule(static,1) clause to the #pragma omp for directive.

However, I am still thinking on the elegant solution of the second issue...

Answer 1

If you don't mind replacing the OpenMP for worksharing construct with a loop that generates tasks instead, you can use OpenMP task to implement both parts of your application.

In the first loop you would create (instead of the loop chunks), tasks that take on the compute load of the iterations. Each iteration of the second loop then also becomes an OpenMP task. The important part then will be to syncronize the tasks between the different phases.

For that you can use task dependencies (introduce with OpenMP 4.0):

#pragma omp task depend(out:A[0])
{ A[0] = a(); }

#pragma omp task depend(in:A[0])
{ b(A[0]); }

Will make sure that task b does not run before task a has completed.

Cheers, -michael

Answer 2

This is probably an extended comment rather than an answer ...

So, you have a two-phase computation. In phase 1 you can compute, independently, each entry in your array A . It is straightforward therefore to parallelise this using an OpenMP parallel for loop. But there is an issue here, naive allocations of work to threads are likely to lead to a (severely ?) unbalanced load across threads.

In phase 2 there is a computation which is not so easily parallelised and which you plan to give to the first thread to finish its share of phase 1.

Personally I'd split this into 2 phases. In the first, use a parallel for loop. In the second drop OpenMP and just have a sequential code. Sort out the load balancing within phase 1 by tuning the arguments to a schedule clause; I'd be tempted to try schedule(guided) first.

If tuning the schedule can't provide the balance you want then investigate replacing parallel for by task -ing.

Do not complicate the code for phase 2 by rolling your own signalling technique. I'm not concerned that the complication will overwhelm you, though you might be concerned about that, but that the complication will fail to deliver any benefits unless you sort out the load balance in phase 1. And when you've done that you don't need to put phase2 inside an OpenMP parallel region.