OpenMP 矩阵乘法花费的时间比预期的要长

Question

I am writing an OpenMP program to multiply two matrices. The idea is that each thread calculates some part of each cell's result. Then, after that, I add those results for each cell to get the result of multiplication.

The problem is that the program takes a lot of time when I use large matrices (512x512 or 1024x1024). Indeed, when I used a matrix of size 1024x1024 using 5 threads, it took 43 seconds, while with 1 thread, it took 14 seconds.

I am thinking It might be the critical section causing huge delays.

Here is the code:

#include <stdio.h>
#include <stdlib.h>
#include <omp.h>

int ** make_array(int n,int m,int f)
{
    int i,j;
    int *linear, **arr;
    linear = malloc(sizeof(int)*m*n);
    arr = malloc(sizeof(int *)*n);
    for(i = 0;i<n;++i) arr[i] = &linear[i*m];
    if(f == 0)
    {
        for(i=0;i<n;++i)
        for(j=0;j<m;++j) arr[i][j] = 0;
        return arr;
    }
    for(i=0;i<n;++i)
        for(j=0;j<m;++j) arr[i][j] = 1+i;
    return arr;
}

void printMat(int **mat, int n)
{
    int i,j;
    for(i = 0; i < n; ++i)
    {
        for(j = 0; j < n;++j)
        {
            printf("%d ",mat[i][j]);
        }
        printf("\n");
    }
}

int main (int argc, char *argv[])
{
    int n;            /// matrix dimension


    scanf("%d", &n);
    double TIME = 0;
    int **a,**b,**c;
    a = make_array(n,n,1);
    b = make_array(n,n,1);
    c = make_array(n,n,0);

    int i,j,k;

    #pragma omp parallel private(i,j,k) shared(a,b,c,TIME)
    {
        double start = omp_get_wtime();
        int **local;
        local = make_array(n,n,0);
        for(i = 0; i < n; ++i)
        {
            for(j = 0; j <n; ++j)
            {
                local[i][j] = 0;
                #pragma omp for schedule(static)
                for(k = 0; k < n; ++k)
                {
                    local[i][j]+= a[i][k] * b[k][j];
                }
            }
        }
        for(i = 0; i <n;++i)
        {
            for(j = 0; j < n; ++j)
            {
                #pragma omp critical
                c[i][j] += local[i][j];
            }
        }
        double end = omp_get_wtime();
        if(TIME < end - start)
        {
            #pragma omp critical
            TIME = end - start;
        }
    }

    printf("%f \n", TIME);
}

Any help would be much appreciated.

Answer 1

This code have lot of problems .

The parallelization method is very inefficient :

For each possible i and j you share a very small work to multiple threads. Moreover, there is an implicit barrier at the end of the parallel for loop. Thus, the communication between threads takes probably much more time than the actual computation.

The critical sections are usually slow (generally implemented using locks). Here you can replace this by atomic operations.

With k threads, the code need k-time more memory and is likely to be memory bound (because of the cache and more data to be filled, not to mention additional page faults that are expensive nowadays).

As a result, you need to rework the parallelization method . You could for example move the #pragma omp for schedule(static) on the i -based loop. Alternatively, you can divide the matrix in chunks and share the work between threads.

Please use BLAS libraries to do matrix multiplications. They are much more optimized than this code.

Here is a list of some other problems:

• There are memory leaks (There are malloc but no free ).
• There is a race condition on the TIME -based condition.
• Jagged array may be inefficient here compared to flat arrays.