带有一组任务的芹菜异常“TimeLimitExceeded”

Question

I have this celery's settings:

WORKER_MAX_TASKS_PER_CHILD = 1
TASK_TIME_LIMIT = 30

When i run group of tasks:

from celery import group, shared_task
from time import sleep

@shared_task
def do_something(arg):
    sleep(60)
    return arg*2

group([do_something.s(i) for i in range(3)]).apply_async()

I'm geting TimeLimitExceeded inside of group and then worker is killed by celery at once. How can i handle it?

Answer 1

According to the documentation :

The soft time limit allows the task to catch an exception to clean up before it is killed: the hard timeout isn't catch-able and force terminates the task.

Answer will be simple: do not use hard-time limits for tasks if you want to catch exception.