I have 20 python scripts and i want to run them in parallel in different bash window, that i can do with the below command and run them in backend parallel:-
python testv.py &
python testv1.py &
python testv2.py &
python testv3.py &
python testv4.py &
python testv5.py &
python testv6.py &
python testv7.py &
python testv8.py &
python testv9.py &
python testv10.py &
python testv11.py &
python testv12.py &
python testv13.py &
python testv14.py &
python testv15.py &
python testv16.py &
python testv17.py &
python testv18.py &
python testv19.py &
python testv20.py &
I converted the above in a bash script :- vaa.sh
#!/bin/bash
python testv.py &
python testv1.py &
python testv2.py &
python testv3.py &
python testv4.py &
python testv5.py &
python testv6.py &
python testv7.py &
python testv8.py &
python testv9.py &
python testv10.py &
python testv11.py &
python testv12.py &
python testv15.py &
python testv16.py &
python testv17.py &
python testv18.py &
python testv19.py &
python testv20.py &
I want to run this for 2 3 hours or say forever till the intervention. How can i achieve this.
I tried to add the vaa.sh in a cronjob for 15 minutes but i want to do it in a way so that as soon as the scripts will finish it should again start whether the total time is 15 mint or 20 min.
while true; do
vaa.sh &;
#wait till the background process is done it will loop again.
wait;
done;
You could do that using multiprocessing
import os
import time
from multiprocessing import Process
def run_program(cmd):
# Function that processes will run
os.system(cmd)
# Creating command to run
commands = ['python testv.py']
commands.extend(['python testv{}.py'.format(i) for i in range(1, 21)])
# Amount of times your programs will run
runs = 1
for run in range(runs):
# Initiating Processes with desired arguments
running_programs = []
for command in commands:
running_programs.append(Process(target=run_program, args=(command,)))
running_programs[-1].daemon = True
# Start our processes simultaneously
for program in running_programs:
program.start()
# Wait untill all programs are done
while any(program.is_alive() for program in running_programs):
time.sleep(1)
If you want to stop executing your programs after some desired time you could do this like so.
desired_time = 2 * 60 * 60 # 2 Hours into seconds
start_time = time.time()
while True:
# Initiating Processes with desired arguments
running_programs = []
for command in commands:
running_programs.append(Process(target=run_program, args=(command,)))
running_programs[-1].daemon = True
# Start our processes simultaneously
for program in running_programs:
program.start()
# Wait untill all programs are done or time has passed
while any(program.is_alive() for program in running_programs) and time.time() - start_time < desired_time:
time.sleep(1)
# If desired time has passed exit main loop
if time.time() - start_time > desired_time:
break
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.