[英]Calling mpirun from python on mpi compiled fortran executable
I got a fortran simulation code that is only able to run in parallel and has to be compiled with mpi ( make mpi=yes
) on at least 4 cores.我得到了一个只能并行运行的 fortran 模拟代码,并且必须在至少 4 个内核上使用 mpi (
make mpi=yes
) 进行编译。 I can run the executable, lets name it "test", without problems when i call mpirun -n 4 ./test
.我可以运行可执行文件,将其命名为“test”,当我调用
mpirun -n 4 ./test
时没有问题。
Now I can generate different input files an process the outputs from python.现在我可以生成不同的输入文件来处理来自 python 的输出。 Hence, I want to execute the above command from python to run several simulations.
因此,我想从 python 执行上述命令来运行几个模拟。 The main problem seems to be, no matter if I use os.system, subprocess.call, .run, .Popen, etc. that for MPI only one process is available (which actually makes sense when python starts one new subprocess):
主要问题似乎是,无论我是否使用 os.system、subprocess.call、.run、.Popen 等,对于 MPI,只有一个进程可用(这在 python 启动一个新子进程时实际上是有意义的):
Eg when using os.system('mpirun -n 4 ./test')
or subprocess.run(['mpirun', '-n', '4', './test'])
, I get the following output:例如,当使用
os.system('mpirun -n 4 ./test')
or subprocess.run(['mpirun', '-n', '4', './test'])
os.system('mpirun -n 4 ./test')
or subprocess.run(['mpirun', '-n', '4', './test'])
,我得到以下输出:
starting MPI-3.1 code.
using 1 nodes with total 1 processors and 1 threads.
node 0: procs= 1 threads= 1 name=my-pcname
>>>>>> General information <<<<<<<
-------------------------------------------------------------------
Started : 22-OCT-2020 18:38:55
Name of host machine :
Current directory : /home/test
Compiled on : Linux (Intel Fortran)
Compiled : without OpenMP
Compiled : with MPI
Linked FFT package : cvecfft_acc
Compiled for : 4 MPI processors
1 OMP threads
Running on : 1 MPI processors
1 OMP threads
1 OMP processors per node
>>>> some more information about simulation parameters...
par.f: Mismatch nproc in par.f and MPI nodes:
Compiled for : 4 MPI processors
Running on : 1 MPI processors
*** STOP *** at location (node 0 ): 3
Interestingly enough I get this output 4 times, which confuses me even more...有趣的是,我得到了 4 次这个输出,这让我更加困惑......
Any ideas on how I could get this to work?关于如何让它发挥作用的任何想法? And sorry, if a similar question was already asked somewhere, I searched for at least an hour and asked colleagues before I decided to post this question here...
抱歉,如果已经有人问过类似的问题,我至少搜索了一个小时并询问了同事,然后才决定在这里发布这个问题......
I use the Intel fortran compiler ifort 19.0.5.281 20190815 together with OpenMPI 4.0.5我使用 Intel fortran 编译器 ifort 19.0.5.281 20190815 和 OpenMPI 4.0.5
It looks strange.看起来很奇怪。 All these messages related to execution environment.
所有这些消息都与执行环境有关。 Are these produced by your code or is it part of your custom
mpirun
script?这些是由您的代码生成的还是您的自定义
mpirun
脚本的一部分?
If I run simple MPI
based case如果我运行简单的基于
MPI
的案例
program main
use mpi
integer error, id, p
call MPI_Init ( error )
call MPI_Comm_size ( MPI_COMM_WORLD, p, error )
call MPI_Comm_rank ( MPI_COMM_WORLD, id, error )
write (*,*) 'Hello: ', id, '/', p
call MPI_Finalize ( error )
end
compiled it with GNU
based chain用基于
GNU
的链编译它
gnuchain/openmpi-4.0.2-gcc-9.2.0
following way: mpif90 -o hello hello.f90
it works.以下方式:
mpif90 -o hello hello.f90
它有效。
And then, if I use simple Python script然后,如果我使用简单的 Python 脚本
import os
os.system('mpirun -np 4 ./hello')
it simply works as expected:它只是按预期工作:
> python ./mpi_run.py
Hello: 1 / 4
Hello: 3 / 4
Hello: 0 / 4
Hello: 2 / 4
Do you, by any chance, run your Python
code itself as an MPI
code?您是否有机
Python
代码本身作为MPI
代码运行?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.