为 django 项目妖魔化 Celery

Question

我已经浪费了很多时间来弄清楚如何使用妖魔化的芹菜在 django 项目中执行任务

版本
Django 版本是 1.3
celery 版本是 3.1

我的目录结构是

project/
   ./manage.py
   ./settings.py

   ./celeryapp/
       ./celeryqueue.py
       ./tasks.py

通过创建此处指定的“/etc/init.d/celeryd”，我已经能够在妖魔化模式下运行 celery

还复制了此处指定的妖魔化芹菜的配置文件

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Where to chdir at start.
CELERYD_CHDIR="/opt/project/"

# Extra arguments to celeryd
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# Name of the celery config module.
CELERY_CONFIG_MODULE="celeryconfig"

# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
CELERYD_USER="celery"
CELERYD_GROUP="celery"

我的 celeryconfig.py 是

BROKER_URL = 'amqp://'

CELERY_RESULT_BACKEND = 'amqp://'

CELERY_IMPORTS = ("celeryapp.tasks",)

我已经创建了 celery 用户并通过执行获得了所需的权限

rabbitmqctl add_user celery celery  
rabbitmqctl add_vhost celeryvhost  
rabbitmqctl set_permissions -p celeryvhost celery ".*" ".*"

我仍然收到 KeyError 说

Received unregistered task of type "mytaskname"

这是回溯

ERROR/MainProcess] Received unregistered task of type 'process_transcoding'.  
The message has been ignored and discarded.  
Did you remember to import the module containing this task?  
Or maybe you are using relative imports?  
Please see http://bit.ly/gLye1c for more information.  
The full contents of the message body was:  
{'utc': True, 'chord': None, 'args': [2202L,u'/opt/ossite/media/videos/Screenshot_from_2013-09-24_150354_1.png', u'/opt/ossite/media/thumbnails/Screenshot_from_2013-09-24_150354_1.jpeg', 'thumbnail'], 'retries': 0, 'expires': None, 'task': 'process_transcoding', 'callbacks': None, 'errbacks': None, 'timelimit': (None, None), 'taskset': None, 'kwargs': {}, 'eta': None, 'id': 'f8468f52-ade3-4405-96d6-21a187583c77'} (381b)  
Traceback (most recent call last):  
File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 457, in  on_task_received  
strategies[name](message, body,
KeyError: 'process_transcoding'

绝望地等待帮助。

提前致谢。

Answer 1

您可以在配置中添加“ --loglevel = DEBUG”：

# Extra arguments to celeryd
CELERYD_OPTS="--time-limit=300 --concurrency=8 --loglevel=DEBUG"

检查日志文件，并在其中看到任何异常情况来编辑问题。 您描述的问题可能是由于多种原因造成的。

Answer 2

您有定期任务吗？ 有时，当我测试某种定期任务并将其从代码中删除时，我会收到类似的错误，但是该任务仍存储在数据库中。 如果您已安装Djcelery，请检查django admin定期任务。 如果您有一个名为“ mytaskname”的任务，则将其删除。

Answer 3

最终，我的老板有足够的带宽和我一起坐着，他找到了确切的问题。

问题是相对进口...

我只需要附加“项目”。 对于每个与项目相关的导入，对于task.py中的所有导入以及导入文件中的所有导入也是如此。 例如

tasks.py  
from celeryapp.celeryqueue import app # replaced by  
from project.celeryapp.celeryqueue import app  # and  

from abc import xyz  # is replced by  
from project.abc import xyz  

xyz.py  
from pqr import stu # is replced by
from project.pqr import stu  

等等...

这是丹尼尔建议的相对进口量的链接