MPI_Scatter，散布对角线元素

Question

I am trying to solve a simple program using the MPI Library.

There is 4*N × 4*N matrix is stored on process 0. Length of each side of the matrix is DIM LEN = 4*N. I need to crate a diagonal Datatype. However, instead of 4*N, the datatype should only cover N consecutive elements on the diagonal.Then I have to use the created datatype to distribute the diagonal of this matrix equally among processes of the MPI program. The total number of processes is 4.

I am not able to receive the diagonal elements properly after scattering.

Non root : 3 
Non root : 1 
2.000000 0.000000 
6.000000 0.000000 
Non root : 2 
4.000000 0.000000 

This is the program

#include "mpi.h"
#include <stdio.h>
#define DIM_LEN 8
int main( int argc, char *argv[] )
{
int myid, numprocs;
double mat[DIM_LEN][DIM_LEN];
double matRcv[DIM_LEN/4];

int i,j;
MPI_Datatype diagonal;

MPI_Init(&argc,&argv);

MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
MPI_Comm_rank(MPI_COMM_WORLD,&myid);

MPI_Type_vector(DIM_LEN/4, 1, DIM_LEN +1 , MPI_DOUBLE, &diagonal);
MPI_Type_commit(&diagonal);

  if (myid == 0){
    printf("Root : %d \n",myid); 
        for ( i=0; i<DIM_LEN; i++)
            for ( j=0; j<DIM_LEN; j++ )
                mat[i][j] = i+j;

        for ( i=0; i<DIM_LEN; i++){
           for ( j=0; j<DIM_LEN; j++ ){
                printf("%lf \t",mat[i][j] );       
           }
           printf("\n");
       }

    /* Scatter the big table to everybody's little table */
}

MPI_Scatter(&mat[0] , 1, diagonal, &matRcv ,1, diagonal, 0, MPI_COMM_WORLD);
MPI_Barrier(MPI_COMM_WORLD);

if(myid != 0){
        printf("Non root : %d \n",myid); 

         for ( i=0; i<DIM_LEN/4; i++)
                printf("%lf ",matRcv[i] );  

           printf("\n");
}

MPI_Type_free(&diagonal);
MPI_Finalize();
return 0;
}

Answer 1

OK. Now that you are correctly striding your diagonal datatype, your datatype should be properly reflecting the memory locations you want in your 2D array mat . Note however, that your diagonal datatype does NOT reflect the memory locations you eventually want in your 1D arrays matRcv . The memory locations you want to receive are the DIM_LEN/4 contiguous slots of memory starting from matRcv[0] . You can either create a 2nd datatype to describe this memory mapping (using MPI_Type_contiguous ) or more easily just call it as DIM_LEN/4 doubles .

Therefore, your MPI_Scatter call should look like:

MPI_Scatter(&mat[0], 1, diagonal, &matRcv, DIM_LEN/4, MPI_DOUBLE, 0, MPI_COMM_WORLD);

Note also that this is a blocking collective, so there is no need for a MPI_Barrier() after calling it; the barrier is implied.

Answer 2

// Without this it would not work.
MPI_Type_create_resized(myType, 0, (DIM_LEN+1)*sizeof(int)*4, &colType);        

// So resize the userdefined datatype whenever Vector Type is used.
MPI_Type_commit(&colType);
MPI_Scatter(&matrix[0], 1, colType, receiveBuffer, DIM_LEN/4, MPI_INT, 0, MPI_COMM_WORLD);