[英]celery task routes not working as expected
I am practicing celery and I want to assign my task to a specific queue however it does not work as expected 我正在练习芹菜,我想将我的任务分配给一个特定的队列,但它不能按预期工作
My __init__.py 我的__init__.py
import os
import sys
from celery import Celery
CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(CURRENT_DIR)
app = Celery()
app.config_from_object('celery_config')
My celery_config.py 我的celery_config.py
amqp = 'amqp://guest:guest@localhost:5672//'
broker_url = amqp
result_backend = amqp
task_routes = ([
('import_feed', {'queue': 'queue_import_feed'})
])
My tasks.py 我的tasks.py
from . import app
@app.task(name='import_feed')
def import_feed():
pass
How I run my worker: 我如何管理我的工人:
celery -A subscriber1.tasks worker -l info
My client's __init__.py
: 我的客户的
__init__.py
:
import os
import sys
from celery import Celery
CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(CURRENT_DIR)
app = Celery()
app.config_from_object('celery_config')
My client's celery_config.py : 我的客户的celery_config.py :
from kombu.common import Broadcast
amqp = 'amqp://guest:guest@localhost:5672//'
BROKER_URL = amqp
CELERY_RESULT_BACKEND = amqp
Then in my client's shell I tried: 然后在我的客户端shell中我试过:
from publisher import app
result = app.send_task('import_feed')
Then my worker got the task?! 然后我的工作人员完成了任务?! Which I expect should not because I assigned that to a specific queue.
我不希望这样,因为我将其分配给特定队列。 I tried in my client the command below and no task has been received by my worker which I expect to have received instead on the first one
我在我的客户端尝试了下面的命令,我的工作人员没有收到任何我希望收到的任务,而不是第一个
result = app.send_task('import_feed', queue='queue_import_feed')
Seems like I misunderstood something in the routing part. 好像我误解了路由部分中的某些内容。 But what I really want is
import_feed
task to run only if the queue_import_feed
queue is specified when send a task 但我真正想要的是
import_feed
任务只有在发送任务时指定了queue_import_feed
队列才能运行
You can change the default queue that the worker processes. 您可以更改工作进程处理的默认队列。
app.send_task('import_feed')
sends the task to celery
queue. app.send_task('import_feed')
将任务发送到celery
队列。
app.send_task('import_feed', queue='queue_import_feed')
sends the task to queue_import_feed
but your worker is only processing tasks in celery
queue. app.send_task('import_feed', queue='queue_import_feed')
将任务发送到queue_import_feed
但您的工作人员只处理celery
队列中的任务。
To process specific queues, use the -Q
switch 要处理特定队列,请使用
-Q
开关
celery -A subscriber1.tasks worker -l info -Q 'queue_import_feed'
Edit 编辑
In order to place a restriction on send_task
such that a worker reacts to import_feed
task only when it's published with a queue, you need to override send_task
on Celery
and also provide a custom AMQP
with a default_queue
set to None
. 为了将限制在
send_task
使得工人反应import_feed
任务,只有当它与队列发布,你需要重写send_task
的Celery
,也提供自定义AMQP
与default_queue
设置为None
。
reactor.py reactor.py
from celery.app.amqp import AMQP
from celery import Celery
class MyCelery(Celery):
def send_task(self, name=None, args=None, kwargs=None, **options):
if 'queue' in options:
return super(MyCelery, self).send_task(name, args, kwargs, **options)
class MyAMQP(AMQP):
default_queue = None
celery_config.py celery_config.py
from kombu import Exchange, Queue
...
task_exchange = Exchange('default', type='direct')
task_create_missing_queues = False
task_queues = [
Queue('feed_queue', task_exchange, routing_key='feeds'),
]
task_routes = {
'import_feed': {'queue': 'feed_queue', 'routing_key': 'feeds'}
}
__init__.py __init__.py
celeree = MyCelery(amqp='reactor.MyAMQP')
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.