简体   繁体   English

如果有 GIL,Python 守护进程线程会被繁忙的父线程阻塞吗?

[英]Will Python Daemon thread be blocked by busy parent thread given there is GIL?

I know Python daemon thread will auto exit if parent thread exit.我知道如果父线程退出,Python 守护进程线程将自动退出。 The reason I ask if because my main thread is sometime working on CPU heavy calculation which blocks the heart beat message.我问的原因是因为我的主线程有时正在处理 CPU 繁重的计算,这会阻止心跳消息。 Other module thinks this service is dead and planning for funeral.其他模块认为此服务已死并正在筹划葬礼。

I want to use some kind of daemon thread(or process?) that is the child of main thread, to send heart beat message.我想使用某种守护线程(或进程?),它是主线程的子线程,来发送心跳消息。

My question is if my main thread is blocking, for example:我的问题是我的主线程是否阻塞,例如:

while True:
   a = a + 1

Will my daemon child thread be blocked as well?我的守护进程子线程也会被阻塞吗? any example or evidence?有什么例子或证据吗? What role does GIL (Global Interpreter Lock) play here? GIL(全局解释器锁)在这里扮演什么角色?

Edit编辑

To @ShadowRanger's point, the following example shows, waiter can log out the the message, given the worker thread is blocking (taking 220% CPU resource)对于@ShadowRanger 的观点,以下示例显示,假设工作线程正在阻塞(占用 220% CPU 资源),服务员可以注销消息

import time
from threading import Thread

class worker(Thread):
    def run(self):
        t = 10
        x = 10
        while True:
            x = t + 1

class waiter(Thread):
    def run(self):
        for x in xrange(100,150):
            print x
            time.sleep(0.5)

def run():
    worker().start()
    waiter().start()

run()

If your main thread is executing Python byte code (not embedded in some expensive long term call to a C extension module that holds the GIL for the length of the call and doesn't return to the interpreter for a long time), then the daemon thread will eventually get a chance to run;如果您的主线程正在执行 Python 字节码(未嵌入对 C 扩展模块的一些昂贵的长期调用中,该模块在调用的长度内保存 GIL 并且长时间不返回到解释器),那么守护进程线程最终将有机会运行; the main thread will be preempted after a while (a matter of milliseconds) and the daemon thread will run while the main thread is blocked.主线程将在一段时间(几毫秒)后被抢占,守护线程将在主线程被阻塞时运行。 As long as your heartbeat interval is measured in seconds, that should be fine.只要您的心跳间隔以秒为单位,那应该没问题。

Only one thread actually runs at any given time, but both will interleave execution.在任何给定时间实际上只有一个线程运行,但两者都会交错执行。

The scenario with long term GIL holding by an extension can come up by accident, to be clear;需要明确的是,通过延期持有长期 GIL 的情况可能是偶然出现的; all extensions default to holding the GIL.所有扩展默认都持有 GIL。 While popular, widely-used extensions like numpy are careful about releasing the GIL when they've got enough work to do, extensions written by less experienced developers (especially those writing in Cython with no knowledge of the GIL) are more likely to hold the GIL for long periods, so if your heartbeat isn't working and extension modules are involved, they're a likely culprit.虽然像numpy这样流行的、广泛使用的扩展在有足够的工作要做时会谨慎地发布 GIL,但由经验不足的开发人员编写的扩展(尤其是那些用 Cython 编写的对 GIL 一无所知的开发人员)更有可能持有GIL 很长一段时间,所以如果你的心跳工作并且涉及扩展模块,它们很可能是罪魁祸首。 A simple while True: a = a + 1 loop would be safe though (assuming a isn't some weirdo extension type);一个简单的while True: a = a + 1循环虽然是安全的(假设a不是某种奇怪的扩展类型); the GIL can be switched easily there. GIL 可以在那里轻松切换。

For safety reasons I'd go with multiprocessing here.出于安全原因,我会在这里进行multiprocessing Your heartbeat response could be a separate process released from chains of GIL.你的心跳响应可能是从 GIL 链中释放出来的一个单独的过程。 Otherwise you'll depend on indeterministic code flow and you may get random errors in least expected moments.否则,您将依赖于不确定的代码流,并且您可能会在最不期望的时刻出现随机错误。

It is probably the IO loop callback inside a thread that blocks the heart beat.可能是线程内部的 IO 循环回调阻止了心跳。 As @ShadowRanger points out, the thread is interleaving, which should not block another thread periodical callback (except the problematic python extension).正如@ShadowRanger 指出的那样,线程是交错的,不应该阻塞另一个线程定期回调(有问题的python 扩展除外)。 In my case, it is probably the ioloop callback queue inside the thread is the culprit.就我而言,可能是线程内的 ioloop 回调队列是罪魁祸首。 No interleaving in ioloop callback queue. ioloop 回调队列中没有交错。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM