What happens after celery's `max_retries` is hit?

Question

I'm familiar with Celery's max_retries option on tasks— it ensures that a task isn't retried more than that number of times.

However, what happens after that max is hit? (eg, if it was hit because an error keeps being thrown) Is the task simply discarded, or if it's due to an error, is the error thrown and the admins emailed (if CELERY_SEND_TASK_ERROR_EMAILS is set to True )?

Answer 1

I believe the task would run its on_failure() method (triggered during final attempt), followed by the after_return() method. You can override either of these if you need to in your Task class. The difference between the previous attempts and the final failure is that the final try does not raise celery.exceptions.Retry; an internal Exception which requeues the task.

If an exception happens during the task with the CELERY_SEND_TASK_ERROR_EMAILS set to True, it should email the list of users. I think this is each time the task fails, so you should get 5 emails if max_retries is set to 5 and the task failed through all 5 (have not tested this though).

It is important to note that the email triggers do not include Celery exceptions (due to the code path), so you will not get an email for Max Retries being reached (or the Retry exception). If you also wanted an email due to max retries, you could override the after_return and check the retries number and/or status.

I believe at this point, the task will no longer be in the queue/messenger, but depending on your settings you should have an entry in the Backend, where you can see that the task failed max_retries times, then finally raised a MaxRetriesExceededError.

If using EAGER tasks, you may wish to set

CELERY_EAGER_PROPAGATES_EXCEPTIONS = True.

Of interest: Recover from task failed beyond max_retries