[英]How to configure sentry to send exception from process spawned by python multiprocessing pools?
I have a script which is run by celery worker which uses Pool from billiard library and I spawned multiple process. 我有一个由celery worker运行的脚本,该脚本使用台球库中的Pool,并且产生了多个进程。 I am trying to use sentry inside those process so that any unhandled/handled exception can be caught. 我试图在这些过程中使用哨兵,以便可以捕获任何未处理/未处理的异常。 Written below is my sample code : 下面写的是我的示例代码:
from configurations import SENTRY_CLIENT
def process_data(data):
try:
s = data/0
except ZeroDivisionError:
print "Sentry must report this."
SENTRY_CLIENT.captureException()
import multiprocessing
from billiard import Pool
POOL_SIZE=multiprocessing.cpu_count()
pool = Pool(POOL_SIZE)
data=[0, 1, 2, 3, 4, 5]
pool.map(process_data, data)
pool.close()
pool.terminate()
SENTRY_CLIENT is defined in configuration file which is defined as : configurations.py SENTRY_CLIENT在配置文件中定义,该文件定义为:configuration.py
from raven import Client
SENTRY_CLIENT = Client("dsn")
One way which I am trying is to pass SENTRY_CLIENT to each process but I am trying to avoid that as of now. 我正在尝试的一种方法是将SENTRY_CLIENT传递给每个进程,但是到目前为止,我正在尝试避免这种情况。 Also as this script is executed by celery worker I had configured sentry for celery and any exception till pool.map()
is well caught by sentry. 另外,由于此脚本是由celery worker执行的,因此我已经为celery配置了pool.map()
以及任何异常,直到pool.map()
被哨兵很好地捕获为止。
Also I tried to print the SENTRY_CLIENT.__dict__
and I got the valid items with correct values. 我也尝试打印SENTRY_CLIENT.__dict__
,我得到了带有正确值的有效项目。 My issue is here that why SENTRY_CLIENT is not sending exceptions to sentry dashboard. 我的问题是,为什么SENTRY_CLIENT不会将异常发送到哨点仪表板。 May be I am missing something in configurations. 可能是我在配置中缺少某些内容。
I finally get the solution by some readings. 我终于通过一些阅读获得了解决方案。 Sentry works on an async event based model and killing a process just after triggering sentry will not ensure the exception is reached to the servers. Sentry在基于异步事件的模型上工作,并且在触发哨兵后立即杀死进程将无法确保服务器达到异常。 Hence we need to add a delay(10s), before killing the process in case of any exception to ensure sentry does it's job. 因此,在任何异常情况下终止进程之前,我们需要添加一个延迟(10s),以确保哨兵能够正常工作。
def process_data(data):
from configurations import SENTRY_CLIENT
try:
s = data/0
except ZeroDivisionError:
print "Sentry must report this."
import time
SENTRY_CLIENT.captureException()
time.sleep(10)
import multiprocessing
from billiard import Pool
POOL_SIZE=multiprocessing.cpu_count()
pool = Pool(POOL_SIZE)
data=[0, 1, 2, 3, 4, 5]
pool.map(process_data, data)
pool.close()
pool.terminate()
As PoloSoares said , you should change a transport instead of adding any delay by sleep. 正如PoloSoares 所说 ,您应该更改传输方式,而不要增加任何睡眠延迟。 Example of valid solution for 6.10.0 version of raven lib: raven lib 6.10.0版本的有效解决方案示例:
import multiprocessing
from billiard import Pool
from raven import Client
from raven.transport.http import HTTPTransport
SENTRY_CLIENT = Client("dsn", transport=HTTPTransport)
def process_data(data):
try:
s = data / 0
except ZeroDivisionError:
print("Sentry must report this.")
SENTRY_CLIENT.captureException()
POOL_SIZE = multiprocessing.cpu_count()
pool = Pool(POOL_SIZE)
data = [0, 1, 2, 3, 4, 5]
pool.map(process_data, data)
pool.close()
pool.terminate()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.