I have a python script which takes in an argument like so.
conda activate base python myScript.py arg1
I'm connected to my server over ssh so there will be no window system. I want to run 5 different with my script in parallel. So like,
#!/bin/bash
conda activate base
for i in arg1 arg2 arg3 arg4 arg5
do
python myScript.py $i
done
But this would only let the code run one after the other. How do I run it in parallel?
With GNU Parallel
parallel python myScript.py ::: arg1 arg2 arg3 arg4 arg5
GNU Parallel can also do the ssh
to one, or more servers for you, so you don't need to login:
parallel -S server 'conda activate base; python myScript.sh' ::: arg1 arg2 arg3 arg4 arg5
Also, add a shebang as the first line of your script like this:
#!/usr/bin/env python
and make your script executable like this:
chmod +x myScript.py
then you can stop telling your shell to use Python every time you want to run it and just use:
./myScript.py arg
You could execute them in the background using &
:
for i in arg1 arg2 arg3 arg4 arg5
do
python myScript.py $i &
# Here ---------------^
done
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.