I have a program, a.out, that will be set up in some sequence of folders, each folder gets an a.out and will produce some results in each folder. I am trying to execute these same programs in parallel. If I am in the folder, I just do ./a.out and it would run. I must execute a.out in its folder because a.out looks for a file inside the current directory. So if I am not in its folder, it won't find that file.
Running these programs is part of another job that is based in the rootDir. I am using MATLAB so I am trying to avoid using cd inside MATLAB since that would recompile the MATLAB code every time I use cd and greatly slow down the code.
I use the MATLAB code to write a CallParallel.sh, in it I have this line:
for i in ${JobsOnThisNode[@]};do echo $i;done | xargs -n1 -P ${SLURM_NTASKS_PER_NODE} sh -c '"$1"' sh;
$1 basically gets this command for each batch of parallel jobs incremented by jname and cname:
cd /mnt/home/thrust2/zf1005/Matlab/GAFit/RunningFolder/$jname/$cname/ && ./a.out
I have tested this code from the rootDir and it successfully runs this program in the other folder. However, when I execute it in the bash script, I get the following errors:
sh: /mnt/home/thrust2/zf1005/Matlab/GAFit/RunningFolder/1/1/: Is a directory
sh: &&: command not found
sh: ./a.out: No such file or directory
If I am understanding it correctly, somehow it does not recognize && and cd somehow only checks if it is a directory instead of actually changing to that directory, and as a result, there is no a.out to be found in the rootDir.
When I try this:
sh '"cd /mnt/home/thrust2/zf1005/Matlab/GAFit/RunningFolder/1/1"'
I get:
sh: "cd /mnt/home/thrust2/zf1005/Matlab/GAFit/RunningFolder/1/1": No such file or directory
sh '"cd /mnt/home/thrust2/zf1005/Matlab/GAFit/RunningFolder/1/1"'
means, interpret "cd /mnt/home/thrust2/zf1005/Matlab/GAFit/RunningFolder/1/1"
using sh
, which does not exist.
to simplify things you can create a runner script that takes the dir as an argument
#! /bin/bash
cd "$1" && /path/to/a.out
and then invoke runner
from xargs
.
BTW, you need only 1 a.out
not 1 per dir.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.