Ordering of cout weird: MPI_Recv before MPI_Send?

Question

I have something like:

if (rank == winner) {
    ballPos[0] = rand() % 128;
    ballPos[1] = rand() % 64;
    cout << "new ball pos: " << ballPos[0] << " " << ballPos[1] << endl;
    MPI_Send(&ballPos, 2, MPI_INT, FIELD, NEW_BALL_POS_TAG, MPI_COMM_WORLD);
} else if (rank == FIELD) {
    MPI_Recv(&ballPos, 2, MPI_INT, winner, NEW_BALL_POS_TAG, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
    cout << "2 new ball pos: " << ballPos[0] << " " << ballPos[1] << endl;
}

But I see in console:

new ball pos: 28 59
2 new ball pos: 28 59

Why isit that the cout after receive prints before the one before send?

Answer 1

These are two different processes doing output at the same time. MPI implementations usually perform standard output redirection for all processes, but it is usually buffered in order to improve performance and to minimise network utilisation. The output from all processes is then sent to mpiexec (or to mpirun , or to whatever other command is used to launch the MPI job) and combined into its standard output. The order in which different chunks/lines from different processes end up in the output is mostly random, so you must not expect that a message from a certain rank would come up first unless some sort of process synchronisatoin is employed.

Also note that the MPI standard does not guarantee that it is possible for all ranks to write to the standard output. The standard provides the MPI_IO predefined attribute key that one can query on MPI_COMM_WORLD in order to obtain the rank of the process that is allowed to perform standard output. Most MPI implementations nowadays perform output redirection on all processes in the MPI job and thus return MPI_ANY_SOURCE for such attribute queries, but this is not guaranteed to always be the case.