Django+Celery integration with periodic tasks

Question

I am a bit confused on how to configure Django+Celery. I have followed what is reported in this guide . Here it is the bunch of configuration I have to write:

BROKER_URL = 'amqp://...'

queue_arguments = {'x-max-length': 1}
CELERY_QUEUES = (
    Queue('queue1', routing_key='queue1', queue_arguments=queue_arguments),
    Queue('queue2', routing_key='queue2', queue_arguments=queue_arguments))

from datetime import timedelta
CELERYBEAT_SCHEDULE = {
    'task1': {
        'task': 'MyProject.tasks.this_is_task_1',
        'schedule': timedelta(seconds=1)
    },
    'task2': {
        'task': 'MyProject.tasks.this_is_task_2',
        'schedule': timedelta(seconds=1)
    }
}


CELERY_ROUTES = {
        'MyProject.tasks.this_is_task_1': {
            'queue': 'queue1',
            'routing_key': 'queue1',
        },
        'MyProject.tasks.this_is_task_2': {
            'queue': 'queue2',
            'routing_key': 'queue2',
        }
}

app = Celery('MyProject')

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'MyProject.settings')
app = Celery('MyProject')

app.conf.update(
    CELERY_TASK_RESULT_EXPIRES=30,
    CELERY_IGNORE_RESULT=True,
)

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

Based on 1 , I should put:

1. The tasks in MyProject/tasks.py
2. The 'app' variable creation and initialization in MyProject/celery.py
3. The configuration variables in MyProject/settings.py

If I do so, i receive the following:

Couldn't apply scheduled task check_block_height: Queue.declare: (406) PRECONDITION_FAILED - inequivalent arg 'x-max-length'for queue 'queue2' in vhost '...': received the value '1' of type 'signedint' but current is none
[2015-11-04 00:30:12,899: DEBUG/MainProcess] beat: Waking up now.

It behaves as if the queue has already been created, but without any option.

If I keep just CELERYBEAT_SCHEDULE and CELERY_ROUTES in settings.py , everything seems to be working. The truth is that the queue configuration is ignored, that is, the CELERY_QUEUE configuration is not used.

Thanks

Answer 1

I solved the problem. The issue was related to the fact that I already had created the queues, which were not limited. By deleting them and restarting, everything worked perfectly by putting: 1. all the configuration in settings.py 2. all the Celery creation in celery.py