在MPI C中为每个进程生成随机数

Question

I am trying to generate random numbers and assign these numbers to the array in every task. I want to make sure the random numbers in different tasks are different. How could I achieve that?

If every MPI task initialize its own array with random numbers (like the code I attached), are these numbers different among task?

I know I could generate a large set of random numbers and broadcast to each task, but this may cause memory issues for large arrays.

Thank you so much in advance for any informations.

void initialize(float* inarray, int n){
    int i;
    for (i=0; i<n; i++){
            inarray[i] = random() / (float)RAND_MAX;
        }
    }
}

void main(int argc, char* argv[]){

    MPI_Comm comm=MPI_COMM_WORLD;
    int numnodes, myid, ierr;
    ierr=MPI_Init(&argc, &argv);
    ierr=MPI_Comm_size(comm, &numnodes);
    ierr=MPI_Comm_rank(comm, &myid);

    int n = 100;
    float *x = malloc(sizeof(float)*n);
    initialize(x, n);

    ierr=MPI_Finalize();
}

Answer 1

Here is my stupid solution: In a loop, one MPI task create random number and send to other tasks sequentially. To me, this is like a manual broadcast but saves a little memory.

    if (myid == 0){
        int i;
        for (i=0; i<n; i++){
            initialize(x, n);
            if (i != myid){
                MPI_Send(&x[0], n, MPI_FLOAT, i, 0, comm);
            }
        }
    }else{
        MPI_Recv(&x[0], n, MPI_FLOAT, 0, 0, comm, MPI_STATUS_IGNORE);
    }

Answer 2

The (pseudo) random number generator has to be seeded, and with a different seed on each rank.

In my environment (CentOS 7) :

#include <stdio.h>
#include <stdlib.h>

#include <mpi.h>

int main(int argc, char *argv[]) {
    int rank;
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    srand(rank+1);
    float f = (float)random() / (float)RAND_MAX;
    printf ("%d: %f\n", rank, f);
    MPI_Finalize();
    return 0;
}

produces

0: 0.840188
1: 0.700976
2: 0.561380
3: 0.916458
4: 0.274746
5: 0.135439
6: 0.486904
7: 0.352761
8: 0.206965
9: 0.565811
10: 0.926345
11: 0.785600
12: 0.632643
13: 0.999498
14: 0.354973
15: 0.215437

Answer 3

So this in fact this is more complex than expected. There is two issue:

First, let say you have 40 mpi processes which all need to draw 10 number, using an uniform distribution between 0 and 1. Then if you take draw the 400 numbers using a single chain, you will get a quite good uniform distribution. But if you draw only 10 number you probably will not. Drawing 10 number out of 40 chain is not strictly equivalent to drawing 400 number from one chain. But this issue will pop only if you need to have a good control of your distribution.

Secondly, let say you want to study a given problem numerically. On thing that you have to test is the convergence of your solver. To do so, you need to see if your solver behave well under refinement of resolution. So what you need is to generate different realization (your random numbers) in a way that you keep the same low frequencies (if you think in Fourier space). To do so you need to be sure that the random number which is at a given place (let say the 327 number of your 400) is always the same, regardless of the amount of processes you use.

Both of those issue shows that simply drawing consecutive number from a chain is not a good solution.

So what you can do is: -use a single chain - 1st process use the first 10 number of the chain. - 2nd process use the 11st to 20th number of the chain. ......

But this means that a given process have to discard a given number of element of the chain. If you use "usual" random number generator, discarding n number is a O(n) operation. Which means that you cant parallelize your problem. What you need is to use a random generator which contains a method to discard n draw in O(1). There is plenty of such method you can find on the web. Or you can code it by yourself (but this is really trying to reinvent the wheel).

Myself I use: https://rdrr.io/cran/sitmo/
But it is for C++, but sure you might be able to find version for C or ompile just this part with C++...