简体   繁体   中英

Celery's @shared_task not working with soft_time_limit

Here's some simple code:

@shared_task(time_limit=10, soft_time_limit=5)
def check_action():
    try:
        __import__("time").sleep(100)
    except celery.exceptions.SoftTimeLimitExceeded as e:
        print("Here", e, type(e))
    except BaseException as e:
        print("There", e, type(e))

It's a shared_task with time_limit and soft_time_limit set up, so I expect Here to be printed after 5 seconds (along with SoftTimeLimitExceeded exception info) and then There to be printed with some hard-timeout exception.

Instead, it doesn't raise SoftTimeLimitExceeded at all, only hard time limit is called, full output is:

celery_1               | [2019-05-31 21:36:36,709: WARNING/MainProcess] There
celery_1               | [2019-05-31 21:36:36,710: WARNING/MainProcess] 10 seconds
celery_1               | [2019-05-31 21:36:36,710: WARNING/MainProcess] <class 'gevent.timeout.Timeout'>

As it can be seen from logs, I use gevent workers, if it's of any help.

UPD #1: the problems seems to be with gevent pool (is I remove --pool=gevent , everything works perfectly).

UPD #2: eventlet pool doesn't seem to support any timeout, even the hard one.

It really is the problem with gevent pool, as mentioned in this pull request.

Question can be considered closed.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM