openmp ordering critical sections

Question

I am trying to create an OpenMP program that will sequentially iterate through a loop. I realize threads are not intended for sequential programs -- I'm trying to either get a little speedup compared to a single thread, or at least keep the execution time similar to a single-threaded program.

Inside my #pragma omp parallel section, each thread computes its own section of a large array and gets the sum of that portion. These all may run in parallel. Then I want the threads to run in order, and each sum is added to the TotalSum IN ORDER. So thread 1 has to wait for thread 0 to complete, and so on. I have this part inside a #pragma omp critical section. Everything runs fine, except that only thread 0 is completing and then the program exits. How can I ensure that the other threads will keep polling? I've tried sleep() and while loops, but it continues to exit after thread 0 completes.

I am not using #pragma omp parallel for because I need to keep track of the specific ranges of the master array that each thread accesses. Here is a shortened version of the code section in question:

//DONE and MasterArray are global arrays. DONE keeps track of all the threads that have completed

int Function()
{
    #pragma omp parallel
    {
    int ID = omp_get_thread_num
    variables: start,end,i,j,temp(array)  (all are initialized here)
    j = 0;

    for (i = start; i < end; i++)
    {
         if(i != start)
               temp[j] = MasterArray[i];
         else
               temp[j] = temp[j-1] + MasterArray[i];
         j++;
    }



    #pragma omp critical
    {     
        while(DONE[ID] == 0 && ERROR == 0) {

           int size = sizeof(temp) / sizeof(temp[0]);           

           if (ID == 0)  {
              Sum = temp[size];
              DONE[ID] = 1;
              if (some situation)
                 ERROR = 1;   //there's an error and we need to exit the function and program
           }
           else if (DONE[ID-1] == 1) {
              Sum = temp[size];
              DONE[ID] = 1;
              if (some situation)
                 ERROR = 1;   //there's an error and we need to exit the function and program
           }
        }
     }
     }
     if (ERROR == 1)
         return(-1);
     else
         return(0);
   }

this function is called from main after initializing the number of threads. It seems to me that the parallel portion completes, then we check for an error. If an error is found, the loop terminates. I realize something is wrong here, but I can't figure out what it is, and now I'm just going in circles. Any help would be great. Again, my problem is that the function exits after only thread 0 executes, but no error has been flagged. I have it running in pthreads too, but that has been simpler to execute. Thanks!

Answer 1

Your attempt of ordering threads with #pragma omp critical is totally incorrect. There can be just one thread in a critical section at any time, and the order in which the threads arrive to the critical section is not determined. So in your code it can happen that eg the thread #2 enters the critical section first and never leaves it, waiting for thread #1 to complete, while the thread #1 and the rest are waiting at #pragma omp critical . And even if some threads, eg thread #0, are lucky to complete the critical section in right order, they will wait on an implicit barrier at the end of the parallel region. In other words, the deadlock is almost guaranteed in this code.

I suggest you do something much simpler and natural to order your threads, namely an ordered section. It should look like this:

#pragma omp parallel
{
    int ID = omp_get_thread_num();

    // Computations done by each thread

    #pragma omp for ordered schedule(static,1)
    for( int t=0; t<omp_get_num_threads(); ++t )
    {
        assert( t==ID );
        #pragma omp ordered
        {
            // Do the stuff you want to be in order
        }
    }
}

So you create a parallel loop with the number of iterations equal to the number of threads in the region. The schedule(static,1) clause makes it explicit that the iterations are assigned one per thread in the order of thread IDs; and the ordered clause allows to use ordered sections inside the loop. Now in the body of the loop you put an ordered section (the block following #pragma omp ordered ), and it will be executed in the order of iterations, which is also the order of thread IDs (as ensured by the assertion).

For more information, you may look at this question: How does the omp ordered clause work?