Python + Celery手动路由

Question

I've been working on getting manual routing set up with Celery, but can't seem to get specific tasks into specific queues. Here's what I've got going on so far pretty much:

CELERY_QUEUES = {
    "default": {
        "binding_key": "default"},
    "medium": {
        "binding_key": "medium"},
     "heavy": {
         "binding_key": "heavy"},
      }

with the routes defined like

CELERY_ROUTES = ({ "tasks.some_heavy_task": {
                   "queue": "heavy",
                   "routing_key": "tasks.heavy"
                  }}, )

and the daemons started like

celeryd -l INFO -c 3 -Q heavy

The "some_heavy_task"'s never get run though. When I remove the routing and just have a default queue I can get them to run. What am I doing wrong here, any suggestions?

Answer 1

I created special celeryconfig file for each tasks, all tasks stored in special queue. Here is example:

CELERY_IMPORTS = ('cleaner_on_celery.tasks',)
CELERYBEAT_SCHEDULE = {
    'cleaner': {
        "task": "cleaner_on_celery.tasks.cleaner",
        "schedule": timedelta(seconds=CLEANER_TIMEOUT),
    },
}
CELERY_QUEUES = {
    "cleaner": {"exchange": "cleaner", "binding_key": "cleaner"}
}
CELERY_DEFAULT_QUEUE = "cleaner"

from celeryconfig import *

You can see in the bottom: I import common celeryconfig module. In this case you can start few celeryd instances. Also I recommend to use it with supervisord, after creating supervisord.conf file for each task you can easy manage them as:

supervisorctl start cleaner
supervisorctl stop cleaner