How do I programmatically pull an item from a Celery queue?

Question

I have a Celery queue where I want to programmatically pull the next item from the queue based on certain triggers, etc.

If I have a task defined as

@task
def my_task():
    do_stuff()

and then place items in the queue via

my_task.apply_async()

Celery will crank through the items in the queue as fast as it can. But what I'd like to be able to do is something like:

my_task.process_next()

Is there any way to control when Celery pulls stuff from a queue?

Answer 1

Celery very reasonably assumes that once something is on the queue you want it processed as soon as possible. there is the countdown argument but I guess that's not what you want.

You could try overriding the Task Class and putting some logic in the __call__ method to check if you really want to process the task and then maybe schedule it for retry if it turns out it's not ready for processing by your schedule.