OpenMP: Can't parallelize nested for loops

Question

I want to parallelize loop with inner loop within it. My Code looks like this:

    #pragma omp parallel for private(jb,ib) shared(n, Nb, lb, lastBlock, jj, W, WT) schedule(dynamic)   //private(ib, jb) shared(n, Nb, lb, lastBlock, jj, W, WT)       //parallel for loop with omp
    for(jb=0; jb<Nb; jb++)          
    {
            int lbh = (jb==Nb-1) ? lastBlock : lb;
            int ip = omp_get_thread_num();

            packWT(a, n, lb, s, jb, colNr, WT[ip], nr); //pack WWT[jb]      


            for(ib=jb; ib<Nb; ib++)
            {
                    int lbv = (ib==Nb-1) ? lastBlock : lb;

                    multBlock_2x4xk(a, n, jj + ib*lb, jj + jb*lb, W+ib*lb*lb, WT[ip], lb, lbv, lbh);    //MULT BLOCK - 2x4xK (W[jb]*W[ib])

            }
    }

I measure time which proc spent on calculating this loops. It is the same for few threads as for one thread. When I change clause

private(jb,ib)

for

private(jb)

Everything is being changed. I mean for few threads proc is calculating faster than for one thread. What is the problem?

Answer 1

The problem is that your inner for loops is not in canonical shape. Therefore openmp fails to parallelize the loops and no speedup can be achieved. The loops need to look like the following picture. Where start, idx and inc are not allowed to be changed during the parallel part of the code.

I think I identified your problem. You are calling these function:

  packWT(a, n, lb, s, jb, colNr, WT[ip], nr); packWT(a, n, lb, s, jb, colNr, WT[ip], nr);
  multBlock_2x4xk(a, n, jj + ib*lb, jj + jb*lb, W+ib*lb*lb, WT[ip], lb, lbv, lbh);

where one argument is the loop variable jb, as jb can be changed inside the function (depending on the function declaration), the compiler decides not to parallelize the loop. To avoid this copy your variable jb to a local variable and hand the local variable to the function.