mpiexec 和 python mpi4py 给出等级 0 和大小 1

Question

I have a problem with running a python Hello World mpi4py code on a virtual machine.

The hello.py code is:

#!/usr/bin/python
#hello.py
from mpi4py import MPI

comm = MPI.COMM_WORLD

size = comm.Get_size()
rank = comm.Get_rank()

print "hello world from process ", rank,"of", size

I've tried to run it using mpiexec and mpirun, but it is not running well. The output:

$ mpirun -c 4 python hello.py 
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1

And from mpiexec:

$ mpiexec -n 4 python hello.py 
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1

They seem not getting rank and size of comm. What can cause this? How to solve it?

mpiexec --version
mpiexec (OpenRTE) 1.6.5

mpirun --version
mpirun (Open MPI) 1.6.5

The system is Ubuntu 14.04 on the Virtal Machine.

Any ideas why? Thanks!

Answer 1

I had the same issue when running the python module emcee . It would give me an error:

"ValueError: Tried to create an MPI pool, but there was only one MPI process available. 
Need at least two."

The solution I found for my particular cluster was to use a different MPI. My code worked with intel-mpi and mpich2 but not openmpi . For this system, all I had to do was switch the MPI. In my PBS script I used module load mpich2 instead of module load openmpi . In this case mpiexec and mpirun worked correctly.

Answer 2

As suggested above and in this question for C, this has to do with having mpirun coming from a different MPI than mpi4py was linked against.

In my case, and I suspect the same is true for many other Python users, this came from having originally installed mpi4py through conda , which pulled a non-system version of MPI into my conda -- ie which mpirun gave a path in my conda environment.

To solve the problem, I ran conda remove mpi4py and then pip install mpi4py , which seemed to rebuild mpi4py against the MPI in the conda environment and solved the issue.