Run Python scripts in parallel and wait for all to finish before executing more parallel scripts

Question

I need to execute my Python scripts in parallel so I use the following batch file for it:

start python C:\myfolder\1.py
start python C:\myfolder\2.py
start python C:\myfolder\3.py

It works fine, but now I need to run three more scripts in parallel AFTER the above first three finish. How can I specify it in the same batch file?

Answer 1

You can easily do this in python without using windows batch commands.

You can use the subprocess library to run external scripts simultaneously and then wait for them all to complete.

processes = []
scripts = [
    r'C:\myfolder\1.py',
    r'C:\myfolder\2.py',
    r'C:\myfolder\3.py',
]
for script in scripts:
    p = subprocess.Popen(['python', script])
    processes.append(p)

for p in processes:
    p.wait()

# Run other processes here

Answer 2

I think it should work like this:

(
    start python C:\myfolder\1.py
    start python C:\myfolder\2.py
    start python C:\myfolder\3.py
) | pause

For an explanation see this answer .