How to submit 2 jobs (to run 2 python scripts in parallel located in 2 different directories) in linux?

Question

I need to run 2 different python scripts: script1 and script2 in parallel without any interaction. They are located as the following way:

dir0 contains the file jobcript.py and 2 directories named dir1, dir2. dir1 contains script1 and dir2 contains script2.

The jobscript.txt has the following lines in it.

#!/usr/bin/python
import subprocess

exit_code1 = subprocess.call(["python", "./dir1/script1", "-o"], shell=False)
exit_code2 = subprocess.call(["python", "./dir2/script2", "-o"], shell=False)

I ran the following command in linux:

$ python jobscript.py -o

But this runs in series. How can I run them in parallel? Solution is much appreciated!

Answer 1

You can get shell to put the process in the background for you:

from subprocess import call
call(["python ./dir1/script1 -o &"], shell=True)
call(["python ./dir2/script2 -o &"], shell=True)

The "&" tells bash to put it in background. If you want python script to wait for result of each script, then you will need to create a thread. I suspect you want to use a bash script instead of python:

#!/bin/bash
python ./dir1/script1 -o &
python ./dir2/script2 -o &

PS: Why call python scripts from python via subprocess in the first place? You can just access that code directly. If you want it to run in parall then multithreading or multiprocessing is your friend.