为什么我使用 MPI_Bcast 和 -O3 编译器标志会收到堆栈粉碎错误，但没有 -O3 一切正常？

Question

I am pretty new to MPI, so apologies if this is simple.

I have some code from a month or two ago that has been working fine, but I decided to go back and revise it. (It was written when I was just starting out, and it's not a performance critical section.) The code basically generates a random graph on one process and then shares the results with all other processes. An excerpt from the baby's-first-steps version follows:

unsigned int *graph;

if (commrank == 0) {
    graph = gengraph(params); //allocates graph memory in function
    if (commsize > 1) {
        for (int k=1; k<commsize; k++) 
            MPI_Send(graph, n*n, MPI_UNSIGNED, k, 0, MPI_COMM_WORLD);
    }
} else {
    MPI_Status recvStatus;
    graph = malloc(sizeof(unsigned int)*n*n);
    MPI_Recv(graph, n*n, MPI_UNSIGNED, 0, 0, MPI_COMM_WORLD, &recvStatus);
}

While obviously naive, this worked just fine for a while, before I chose to go back and do it in what I thought was the proper fashion:

if (commrank == 0) {
    graph = gengraph(params);
    MPI_Bcast(graph, n*n, MPI_UNSIGNED, 0, MPI_COMM_WORLD);
} else {
    graph = malloc(sizeof(unsigned int)*n*n);
    MPI_Bcast(graph, n*n, MPI_UNSIGNED, 0, MPI_COMM_WORLD);
}

The problem is, I keep getting when "stack smashing" errors in the second version when I compile with -O3 optimization, though it works fine when compiled unoptimized. Note that I have checked the graph allocation function multiple times and debugged it, and it appears to be fine. I have also debugged the second version, and it appears to work fine. The crash occurs later when I try to free the graph memory. (Note that this is not a double free error, and, again, it works fine in the naive implementation and has for some time.)

One final wrinkle: The first version also fails if, instead of using the recvStatus variable, I instead use MPI_STATUS_IGNORE . And, again, this only fails with -O3.

Any thoughts would be greatly appreciated. If it's any help, I'm using mpicc on top of gcc 7.5.0, but I imagine I'm doing something stupid rather than encountering a compiler problem.

Answer 1

I changed the mpicc compiler to Clang and used Address Sanitizer, per the suggestion of @hristo-iliev, and found an error in a subsequent MPI call (a recv with the wrong count size). This led to the undefined behavior. Notably, the address sanitizer pinpointed location of the error quite clearly, while valgrind only gave rather opaque indications that something was going on in MPI (as, well, it always does).

Apologies to the StackOverflow community for this, as the code above was not the culprit (not entirely surprising). It was just some standard undefined behavior due to sloppiness.