[英]KeyboardInterrupt with Python multiprocessing.Pool
I want to write a service that launches multiple workers that work infinitely and then quit when main process is Ctrl+C'd.我想编写一个服务,启动多个无限工作的工作人员,然后在主进程为 Ctrl+C 时退出。 However, I do not understand how to handle Ctrl+C correctly.
但是,我不明白如何正确处理 Ctrl+C。
I have a following testing code:我有以下测试代码:
import os
import multiprocessing as mp
def g():
print(os.getpid())
while True:
pass
def main():
with mp.Pool(1) as pool:
try:
s = pool.starmap(g, [[]] * 1)
except KeyboardInterrupt:
print('Done')
if __name__ == "__main__":
print(os.getpid())
main()
When I try to Ctrl+C it, I expect process(es) running g
to just receive SIGTERM
and silently terminate, however, I receive something like that instead:当我尝试 Ctrl+C 它时,我希望运行
g
的进程只接收SIGTERM
并静默终止,但是,我收到了类似的东西:
Process ForkPoolWorker-1:
Done
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.8/multiprocessing/pool.py", line 51, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "test.py", line 8, in g
pass
KeyboardInterrupt
This obviously means that parent and children processes both raise KeyboardInterrupt
from Ctrl+C, further suggested by tests with kill -2
.这显然意味着父进程和子进程都从 Ctrl+C 引发
KeyboardInterrupt
,使用kill -2
的测试进一步建议。 Why does this happen and how to deal with it to achieve what I want?为什么会发生这种情况以及如何处理它以实现我想要的?
The signal that triggers KeyboardInterrupt
is delivered to the whole pool.触发
KeyboardInterrupt
的信号被传递到整个池。 The child worker processes treat it the same as the parent, raising KeyboardInterrupt
.子工作进程将其视为与父进程相同,提高
KeyboardInterrupt
。
The easiest solution here is:这里最简单的解决方案是:
SIGINT
handling in each worker on creationSIGINT
处理KeyboardInterrupt
KeyboardInterrupt
时终止工作人员You can do this easily by passing an initializer
function that the Pool
runs in each worker before the worker begins doing work:您可以通过在工作人员开始工作之前传递
Pool
在每个工作人员中运行的initializer
函数来轻松完成此操作:
import signal
import multiprocessing as mp
# Use initializer to ignore SIGINT in child processes
with mp.Pool(1, initializer=signal.signal, initargs=(signal.SIGINT, signal.SIG_IGN)) as pool:
try:
s = pool.starmap(g, [[]] * 1)
except KeyboardInterrupt:
print('Done')
The initializer
replaces the default SIGINT
handler with one that ignores SIGINT
in the children, leaving it up to the parent process to kill them. initializer
程序用忽略子进程中的SIGINT
的处理程序替换默认的SIGINT
处理程序,将其留给父进程杀死它们。 The with
statement in the parent handles this automatically (exiting the with
block implicitly calls pool.terminate()
), so all you're responsible for is catching the KeyboardInterrupt
in the parent and converting from ugly traceback to simple message.父级中的
with
语句会自动处理此问题(退出with
块隐式调用pool.terminate()
),因此您只需在父级中捕获KeyboardInterrupt
并将丑陋的回溯转换为简单的消息。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.