如何从已在 SLURM 上使用 srun 启动的 fortran 代码中运行并行程序？

Question

I think my question is pretty specific and niche, and couldn't find an answer anywhere else.

I have a parallel code in Fortran (using MPI), and I would like a subroutine on each individual processor to call another (in principle serial) program during runtime. I do this with EXECUTE_COMMAND_LINE . Now it turns out the other code I'm calling is also parallelized, with no possibility of producing a purely serial version without MPI. In my SLURM file, the cluster is set up such that I have to use srun, so srun./mycode < input.in > output.out calls my code. In the 3rd party code, however, the easiest way to specify the number of cores is to use the provided launcher, which itself uses mpirun to launch the right number of nodes.

In principle, it is possible to run the 3rd party code without mpirun, in which case it should launch a "serial" version (parallel version but on a single core). However, as my code is already being run with srun, it looks like this is triggering the parallel version of the 3rd party software to run on multiple processors, which is ruining what I'm trying to do with this. If I use the normal launcher that calls mpirun to invoke the 3rd party code, everything hangs because mpirun is waiting for the first instance of srun to complete, which it never will.

Is there any way I can specify to the 3rd party code (that doesn't have a flag to specify this explicitly without invoking mpirun) to run on a single processor? Perhaps an environment variable I can set, or a way of using EXECUTE_COMMAND_LINE that would specify the number of cores to run the command on? Or even a way to make multiple mpirun commands interact with preventing each other from running?

I use Intel compilers and MPI versions for everything.

Answer 1

A colleague found one way to do this for anyone struggling:

call execute_command_line("bash -lc 'env -i PATH=/usr/bin/:/bin mpirun -n 2 ./bin/slave &> slave.out &'", wait=.false.)

Executed from within the calling fortran code.