[英]Why does Python hang when running mpirun within a subprocess?
我有一个使用mpi4py的非常简单的MPI脚本
# mpitest.py
from mpi4py import MPI
import time
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
time.sleep(100)
如果我用mpirun正常运行,一切正常
$ mpirun --np 4 python mpitest.py # just fine
但是,如果我使用子进程模块从Python中运行它,那么事情就会运行,但是我的解释器变得非常迟缓
>>> import subprocess
>>> proc = subprocess.Popen(['mpirun', '--np', '2', 'python', 'mpitest.py'])
我已经尝试过像shell=True
这样的关键字参数。
我使用最新的Miniconda for Linux安装了Python,mpi4py和mpich
mrocklin@carbon:~/workspace/play$ conda list | grep mpi
mpi4py 2.0.0 py36_2
mpich2 1.4.1p1 0
https://conda.io/miniconda.html
mrocklin@carbon:~/workspace/play$ conda create -n test-mpi python=3.6 mpi4py
Fetching package metadata .........
Solving package specifications: .
Package plan for installation in environment /home/mrocklin/Software/anaconda/envs/test-mpi:
The following NEW packages will be INSTALLED:
mpi4py: 2.0.0-py36_2
mpich2: 1.4.1p1-0
openssl: 1.0.2l-0
pip: 9.0.1-py36_1
python: 3.6.2-0
readline: 6.2-2
setuptools: 27.2.0-py36_0
sqlite: 3.13.0-0
tk: 8.5.18-0
wheel: 0.29.0-py36_0
xz: 5.2.3-0
zlib: 1.2.11-0
xz-5.2.3-0.tar 100% |################################| Time: 0:00:00 3.79 MB/s
zlib-1.2.11-0. 100% |################################| Time: 0:00:00 5.68 MB/s
#
# To activate this environment, use:
# > source activate test-mpi
#
# To deactivate an active environment, use:
# > source deactivate
#
mrocklin@carbon:~/workspace/play$ source activate test-mpi
(test-mpi) mrocklin@carbon:~/workspace/play$ python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> proc = subprocess.Popen(['mpirun', '--np', '2', 'python', 'mpitest.py'])
这可以通过将stdin=subprocess.DEVNULL
关键字添加到subprocess.Popen
调用来解决,如下所示:
>>> proc = subprocess.Popen(['mpirun', '--np', '2', 'python', 'mpitest.py'],
stdin=subprocess.DEVNULL)
事实证明, mpirun
稍微劫持了stdin管道,这使得许多发往python
进程的键击都没有到达。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.