[英]concurrent.futures ThreadPoolExecutor fails to wait?
I am trying to get a script performance improved, using ThreadPoolExecutor from concurrent.futures. 我试图通过并发使用ThreadPoolExecutor来提高脚本性能。 I am launching some external python scripts via Popen and encapsulating them as a future objects, but these objects enter the callback function as finished, but I can see them running on my machine (they run for quite some minutes).
我通过Popen启动了一些外部python脚本,并将它们封装为将来的对象,但是这些对象完成后进入了回调函数,但是我可以看到它们在我的机器上运行(它们运行了相当长的时间)。 The code looks like this:
代码如下:
with futures.ThreadPoolExecutor(max_workers=4) as executor:
p1 = executor.submit(subprocess.Popen([myotherPythonscript1], stdout = subprocess.PIPE))
p1.add_done_callback(process_result)
p2 = executor.submit(subprocess.Popen([myotherPythonscript2], stdout = subprocess.PIPE))
p2.add_done_callback(process_result)
def process_result( future ):
logger.info( "Seeding process finished...")
I also tried different approaches with running() and wait() future functions, but with the same results. 我还尝试过使用running()和wait()将来的函数使用不同的方法,但是结果相同。 Future objects are marked as already done, but in fact they are still running.
将来的对象被标记为已完成,但实际上它们仍在运行。 Am I missing something?
我想念什么吗?
Thanks, 谢谢,
you can't just pass the result of Popen
to your executor, you have to pass a callable. 您不能只是将
Popen
的结果传递给执行程序,而必须传递一个callable。
>>> from concurrent.futures import *
>>> from subprocess import *
>>> executor = ThreadPoolExecutor(4)
>>> future = executor.submit(Popen(['echo', 'test'], stdout=PIPE))
>>> future.exception()
TypeError("'Popen' object is not callable",)
this on the other hand works: 另一方面,它可以工作:
from concurrent.futures import *
from subprocess import *
def make_call():
process = Popen(['echo', 'test'], stdout=PIPE)
return process.communicate()[0]
executor = ThreadPoolExecutor(4)
future = executor.submit(make_call)
print(future.result())
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.