简体   繁体   中英

SQS with Celery configuration

I was trying to setup Amazon SQS for Celery and I have the below configuration:

BROKER_BACKEND = "SQS"
BROKER_TRANSPORT_OPTIONS = {
    'region': 'us-east-1',
}
AWS_ACCESS_KEY_ID = # access id
AWS_SECRET_ACCESS_KEY = # secret access key
os.environ.setdefault("AWS_ACCESS_KEY_ID", AWS_ACCESS_KEY_ID)
os.environ.setdefault("AWS_SECRET_ACCESS_KEY", AWS_SECRET_ACCESS_KEY)

BROKER_URL = 'sqs://'

CELERY_IMPORTS = ("tasks", )
CELERY_TASK_RESULT_EXPIRES = 300

CELERY_DEFAULT_QUEUE = #queue name
CELERY_DEFAULT_EXCHANGE = CELERY_DEFAULT_QUEUE
CELERY_DEFAULT_EXCHANGE_TYPE = CELERY_DEFAULT_QUEUE
CELERY_DEFAULT_ROUTING_KEY = CELERY_DEFAULT_QUEUE
CELERY_QUEUES = {
    CELERY_DEFAULT_QUEUE: {
        'exchange': CELERY_DEFAULT_QUEUE,
        'binding_key': CELERY_DEFAULT_QUEUE,
    }
}

In my SQS configuration on the AWS account, I have a queue with the name written in CELERY_DEFAULT_QUEUE . When I run this locally, everything works...but for some reason it creates another queue on SQS with the name format <user_id>-celery-pidbox . Something like this: MyUser-MacBook-Pro-local-celery-pidbox .

Is this normal? Why would it be creating another queue when I have a queue created with the name specified? Otherwise, its working, not sure if that other queue is required or I missed something? Any help is appreciated, I could not find this in the docs.

EDIT

Turns out this is normal. For some reason django-celery does this, it creates a queue for each box that you have accessing the queue you want to access. They will fix this in a future release. If somebody knows how to fix this temporarily, please let me know, thanks!

This is actually a good behavior so you can monitor which instances (IPs or local names) are accessing your SQS account. It is just one request, so it won't cost you anything.

You need to set these:

 CELERY_ENABLE_REMOTE_CONTROL = False 
 CELERY_SEND_EVENTS = False

To disable that.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM