[英]Processes in multiprocessing.Pool not reporting runtime errors
I have a pool of processes in python using from multiprocessing import Pool
. 我from multiprocessing import Pool
使用了python中的进程from multiprocessing import Pool
。 I pass to those processes different functions to be ran, calling the Pool.apply_async
function. 我将要运行的不同功能传递给那些进程,调用Pool.apply_async
函数。 If I introduce an error in one of those functions (ie: a line of code such as 5/0
), the expected command line report ZeroDivisionError: integer division or modulo by zero
never shows up, and the program never terminates. 如果我在其中一个函数中引入错误(即: 5/0
之类的代码行),则预期的命令行报告ZeroDivisionError: integer division or modulo by zero
永远不会显示,并且程序永远不会终止。 Even if I introduce a callback function in the call to Pool.apply_async
, the callback function never gets called if the function that the process has to execute has an error on it. 即使我在对Pool.apply_async
的调用中引入了回调函数,但如果进程必须执行的函数出现错误,则也不会调用该回调函数。
How can I have those processes in the pool report errors and terminate if something goes wrong? 如何在池中使那些进程报告错误并在出现问题时终止?
You have to actually try to get
the result from the AsyncResult
returned by apply_async
(or map_async
) for the exception to be raised in the parent. 您实际上必须尝试从apply_async
(或map_async
)返回的AsyncResult
get
结果,以便在父级中引发异常。
def func():
raise Exception("We failed")
...
result = pool.apply_async(func, args=(arg))
time.sleep(2)
result.get() # Exception only gets raised here
Any callback
you provide is only executed if the function returns successfully. 您提供的任何callback
仅在函数成功返回时执行。 It gets skipped if it raises an exception. 如果引发异常,它将被跳过。
In Python 3.2+, the error_callback
keyword argument was introduced, which allows you to pass a callback that gets executed if an exception is raised in the worker, but you can't do that in Python 2.x. 在Python 3.2+中,引入了error_callback
关键字参数,该参数允许您传递一个回调,如果在worker中引发异常,该回调将被执行,但是您不能在Python 2.x中做到这一点。 What you can do is use wrap your worker function in a try
/ except
block that returns any exception raised in the worker, rather than raising it: 您可以执行的操作是将工作器函数包装在try
/ except
块中,该函数返回在工作器中引发的任何异常,而不是引发该异常:
def func():
try:
raise Exception("We failed")
except Exception as e:
return e
Then you can have a normal callback function that checks to see if an Exception
was returned: 然后,您可以使用普通的回调函数来检查是否返回了Exception
:
def callback(result):
if isinstance(result, Exception):
# Do whatever you need to do to clean up and exit
else:
# Function was successful
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.