C, openMPI: What is the best way to distribute blocks of data from each process to all other processes?

Question

in my mpi program each process has or works with a block of data:

char *datablock;

the blocks are of similar but not identical size.

What is the best way (which functions to use and how) to distribute those blocks from each process to each other process? In the end i want each process to have (maybe) an array of all blocks:

char **blockarray;

so that

*blockarray[i] // for i in [0... number_of_processes-1]

is the former block of the ith process. BUT it is not the order that matters, "i" does not have to be the process id and the order can differ on each process (if this is faster)! I just want the fastest way to get each block on each thread.

Answer 1

You should use MPI_Allgatherv Have a look at the documentation here: http://mpi.deino.net/mpi_functions/MPI_Allgatherv.html

With this function you can distribute data from each process to all other processes.

You don't need your char *datablock; You just have the char **blockarray; Each time when you need to synchronize the data you call MPI_Allgatherv . Pseudo code:

id = process_id
recvcounts = [length(blockarray[0]), length(blockarray[1]), ...]
disply = [0, recvcounts[1], recvcounts[0]+recvounts[1], ...]

MPI_Allgatherv(*blockarray[id], length(*blockarray[id]), sendtype, *blockarray, *recvcounts, 
               *displs, recvtype, comm);