[英]How to use python script to start other python scripts and check whether all “child scripts” are finished?
Basically, I want to use a script to start other scripts in different directories and do follow up works until all the "child scripts" are finished. 基本上,我想使用一个脚本在不同目录中启动其他脚本,并进行后续工作,直到完成所有“子脚本”。 I use multiprocessing.Pool to put all the child process together and use wait() to wait until all finished.
我使用multiprocessing.Pool将所有子进程放在一起,并使用wait()等待所有操作完成。 Here is my script:
这是我的脚本:
import os, shutil, subprocess,sys, multiprocessing
rootdir=os.getcwd()
argument=[]
def Openpy(inputinfo):
dirpath, filename= inputinfo
os.chdir(dirpath)
return subprocess.Popen( [ 'python', filename ] )
if __name__=="__main__":
for dirpath, dirname , filenames in os.walk(rootdir):
for filename in filenames:
if filename=='Test.py':
argument.append( [ dirpath, filename ] )
print argument
po=multiprocessing.Pool()
r=po.map_async(Openpy, argument)
po.close()
r.wait()
print 'Child scripts are finished'
The child processes start normally and finish in one minute. 子进程正常启动,并在一分钟内完成。 However, it seems that those child processes do not return its "finished message" to the parent process.
但是,这些子进程似乎没有将其“完成的消息”返回给父进程。 It did not show that "Child scripts are finished".
它没有显示“子脚本已完成”。 What can I do to change my code so that it can detect the end of the child processes properly?
我该怎么做才能更改代码,以便它可以正确检测到子进程的结束?
All help is appreciated! 感谢所有帮助!
2013/9/1 9:16 UTC+8 2013/9/1 9:16 UTC + 8
JF Sebastian's answer is being accepted. 塞巴斯蒂安·JF的答案被接受。 Run successfully and elegant usage of list comprehension makes the whole stuff more readable.
成功运行,优雅地使用列表理解使整个内容更具可读性。
Try swapping the location of r.wait() and po.close() 尝试交换r.wait()和po.close()的位置
if __name__=="__main__":
for dirpath, dirname , filenames in os.walk(rootdir):
for filename in filenames:
if filename=='Test.py':
argument.append( [ dirpath, filename ] )
print argument
po=multiprocessing.Pool()
r=po.map_async(Openpy, argument)
r.wait()
po.close()
It might not be your problem, but it looks like maybe you are closing the pool before the result is evaluated. 可能不是您的问题,但看起来您可能正在关闭池,然后才评估结果。
Popen()
returns immediately so your parent process finishes long before your child scripts ie, look for the "finished message" at the very beginning of the output. Popen()
立即返回,因此您的父进程在子脚本之前就完成了很长时间,即,在输出的开始处查找“完成的消息”。
You don't need multiprocessing
to run subprocesses concurrently: 您不需要
multiprocessing
即可同时运行子流程:
import os
import sys
from subprocess import Popen
# run all child scripts in parallel
processes = [Popen([sys.executable, filename], cwd=dirpath)
for dirpath, dirname , filenames in os.walk('.')
for filename in filenames
if filename == 'Test.py']
# wait until they finish
for p in processes:
p.wait()
print("all done")
See also Python threading multiple bash subprocesses? 另请参见Python线程化多个bash子进程?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.