无法通过MPI_send发送整个矢量

Question

I have been trying to learn MPI. and when i try to run the following code i get the wrong output.

if (world_rank == 0){

    vector<vector<double> > n(4,vector<double>(4));

    srand(time(NULL));

    for(int i=0; i<4 ;i++){
        for(int j=0;j<4;j++){
            n[i][j] = (double)rand()/RAND_MAX;
            cout << n[i][j] << " ";
        }
        cout << endl;
    }
    MPI_Send((void*)&n[0][0],16*sizeof(double),MPI_BYTE,1,0,MPI_COMM_WORLD);
}else{
    MPI_Status status;

    vector<vector<double> > n(4,vector<double>(4));

    MPI_Probe(0,0,MPI_COMM_WORLD,&status);

    int size;

    MPI_Get_count(&status,MPI_BYTE,&size);

    cout << endl << size << endl;

    MPI_Recv((void*)&n[0][0],16*sizeof(n[0][0]),MPI_BYTE,0,0,MPI_COMM_WORLD,MPI_STATUS_IGNORE);

    cout.flush();
    cout << endl;

    for(int i=0; i<4 ;i++){
        for(int j=0;j<4;j++){
            cout << n[i][j] << " ";
        }
        cout << endl;
    }
}

i get all the double values except the last 3. like this.

0.824468 0.752417 0.757125 0.470763 
0.251683 0.703306 0.157991 0.764423 
0.815327 0.0402807 0.897109 0.313816 
0.997203 0.796665 0.0522305 0.797733 

128

0.824468 0.752417 0.757125 0.470763 
0.251683 0.703306 0.157991 0.764423 
0.815327 0.0402807 0.897109 0.313816 
0.997203 0 0 0

can anyone tell me why this is happening? i ran the same code around a hundred times and still get the same output (of course with different values) but the last three always are 0.

but when i changed the size from 16 to 19 i get all the values.

i also have another doubt. some times the outputs (values from node 0 and 1) get overlapped. can anyone tell me how to stop that or at least explain why that happens. i mean even though send and recv are blocking functions. how can the node 1's output get printed before node 0's

Answer 1

Your definition of the 2D data n as vector<vector<double> > makes it non-contiguous in memory. Therefore, you cannot transmit it simply using MPI (there are ways for doing it, but you'd better just making the memory contiguous instead).

To have your memory contiguous, you can declare your n like this (not tested):

vector<double> ndata(4*4); //contiguous storage of the actual data
vector<double*> n(4);      //vector of pointers to access the actual data
for (int i=1; i<4; i++)    //initialisation of the pointers to the data
    n[i] = &ndata[4*i];

Of course, there are better ways of defining a contiguous storage for multidimensional arrays in C++, but this is just a quick fix for your immediate problem. See for example this answer for a better structure.

And BTW, your MPI_Send() and MPI_Recv() calls should use 4*4 MPI_DOUBLE instead of 4*4*sizeof(double) MPI_BYTE .