[英]Cannot get working Django Celery Kombu using tutorial
嘗試使其正常工作:使用教程http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html的 Ubuntu 14.04上的Django 1.6.11 + Celery 3.1.19 + Kombu 3.0.33所以我從這里完全重復了項目結構: https : //github.com/celery/celery/tree/3.1/examples/django/
當我從Django項目根目錄運行命令時:/ var / www / engine $ celery worker -l INFO -B
在我的應用程序vk_wall中,我具有帶有添加任務的task.py文件。
我得到一個錯誤:
graph@engine:/var/www/engine$ celery worker -l INFO -B
[2016-01-17 18:41:41,159: WARNING/MainProcess] /usr/local/lib/python3.4/dist-packages/celery/apps/worker.py:161: CDeprecationWarning:
Starting from version 3.2 Celery will refuse to accept pickle by default.
The pickle serializer is a security concern as it may give attackers
the ability to execute any command. It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.
If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
You must only enable the serializers that you will actually use.
warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
-------------- celery@engine v3.1.19 (Cipater)
---- **** -----
--- * *** * -- Linux-3.13.0-24-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: default:0x7f9c6495cc18 (.default.Loader)
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results: disabled
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ----
--- ***** ----- [queues]
-------------- .> celery exchange=celery(direct) key=celery
[tasks]
[2016-01-17 18:41:41,187: INFO/Beat] beat: Starting...
[2016-01-17 18:41:41,212: ERROR/Beat] Removing corrupted schedule file 'celerybeat-schedule': error(13, 'Permission denied')
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/kombu/utils/__init__.py", line 323, in __get__
return obj.__dict__[self.__name__]
KeyError: 'scheduler'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 376, in setup_schedule
self._store = self._open_schedule()
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 366, in _open_schedule
return self.persistence.open(self.schedule_filename, writeback=True)
File "/usr/lib/python3.4/shelve.py", line 239, in open
return DbfilenameShelf(filename, flag, protocol, writeback)
File "/usr/lib/python3.4/shelve.py", line 223, in __init__
Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
File "/usr/lib/python3.4/dbm/__init__.py", line 94, in open
return mod.open(file, flag, mode)
_gdbm.error: [Errno 13] Permission denied
[2016-01-17 18:41:41,219: ERROR/Beat] Process Beat
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/kombu/utils/__init__.py", line 323, in __get__
return obj.__dict__[self.__name__]
KeyError: 'scheduler'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 376, in setup_schedule
self._store = self._open_schedule()
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 366, in _open_schedule
return self.persistence.open(self.schedule_filename, writeback=True)
File "/usr/lib/python3.4/shelve.py", line 239, in open
return DbfilenameShelf(filename, flag, protocol, writeback)
File "/usr/lib/python3.4/shelve.py", line 223, in __init__
Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
File "/usr/lib/python3.4/dbm/__init__.py", line 94, in open
return mod.open(file, flag, mode)
_gdbm.error: [Errno 13] Permission denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/billiard/process.py", line 292, in _bootstrap
self.run()
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 545, in run
self.service.start(embedded_process=True)
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 464, in start
humanize_seconds(self.scheduler.max_interval))
File "/usr/local/lib/python3.4/dist-packages/kombu/utils/__init__.py", line 325, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 504, in scheduler
return self.get_scheduler()
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 499, in get_scheduler
lazy=lazy)
File "/usr/local/lib/python3.4/dist-packages/celery/utils/imports.py", line 53, in instantiate
return symbol_by_name(name)(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 358, in __init__
Scheduler.__init__(self, *args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 185, in __init__
self.setup_schedule()
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 378, in setup_schedule
self._store = self._destroy_open_corrupted_schedule(exc)
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 372, in _destroy_open_corrupted_schedule
return self._open_schedule()
File "/usr/local/lib/python3.4/dist-packages/celery/beat.py", line 366, in _open_schedule
return self.persistence.open(self.schedule_filename, writeback=True)
File "/usr/lib/python3.4/shelve.py", line 239, in open
return DbfilenameShelf(filename, flag, protocol, writeback)
File "/usr/lib/python3.4/shelve.py", line 223, in __init__
Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
File "/usr/lib/python3.4/dbm/__init__.py", line 94, in open
return mod.open(file, flag, mode)
_gdbm.error: [Errno 13] Permission denied[2016-01-17 18:41:41,227: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-01-17 18:41:41,252: INFO/MainProcess] mingle: searching for neighbors
[2016-01-17 18:41:42,272: INFO/MainProcess] mingle: all alone
[2016-01-17 18:41:42,302: WARNING/MainProcess] celery@engine ready.
[2016-01-17 18:42:45,778: ERROR/MainProcess] Received unregistered task of type 'vk_wall.tasks.mul'.
The message has been ignored and discarded.
Did you remember to import the module containing this task?
Or maybe you are using relative imports?
Please see http://bit.ly/gLye1c for more information.
The full contents of the message body was:
{'expires': None, 'errbacks': None, 'callbacks': None, 'args': (2, 5), 'kwargs': {}, 'utc': True, 'retries': 0, 'eta': '2016-01-17T15:42:55.746877+00:00', 'task': 'vk_wall.tasks.mul', 'id': '38c3c3d3-3b53-4c58-a308-d41ed24ea613', 'timelimit': (None, None), 'chord': None, 'taskset': None} (302b)
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/celery/worker/consumer.py", line 455, in on_task_received
strategies[name](message, body,
KeyError: 'vk_wall.tasks.mul'
[2016-01-17 18:42:45,786: ERROR/MainProcess] Received unregistered task of type 'vk_wall.tasks.add'.
The message has been ignored and discarded.
Did you remember to import the module containing this task?
Or maybe you are using relative imports?
Please see http://bit.ly/gLye1c for more information.
The full contents of the message body was:
{'expires': None, 'errbacks': None, 'callbacks': None, 'args': (2, 2), 'kwargs': {}, 'utc': True, 'retries': 0, 'eta': None, 'task': 'vk_wall.tasks.add', 'id': 'cc04ddae-024e-4884-9a10-7a7066206fa9', 'timelimit': (None, None), 'chord': None, 'taskset': None} (264b)
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/celery/worker/consumer.py", line 455, in on_task_received
strategies[name](message, body,
KeyError: 'vk_wall.tasks.add'
我看到芹菜跳動問題,也在回溯中我看到芹菜可以找到添加,mul,get_wallposts_by_owner_id任務,但無法執行它們(在我的觀點中。 2,5),queue ='celery',countdown = 10)res_add = add.delay(2,2))
我還安裝了django-celery以在管理面板中查看任務。
順便說一句:在初學者教程http://docs.celeryproject.org/en/latest/getting-started/next-steps.html期間,我能夠讓celery工作並執行任務,但是現在需要django才能訪問任務中的模型。
UPD:我的settings.py Django文件
BROKER_URL = 'django://'
#: Only add pickle to this list if your broker is secured
#: from unwanted access (see userguide/security.html)
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_RESULT_EXPIRES=3600,
#CELERY_RESULT_BACKEND='amqp',
CELERY_RESULT_BACKEND='djcelery.backends.database:DatabaseBackend',
#CELERY_RESULT_BACKEND='djcelery.backends.cache:CacheBackend',
CELERY_RESULT_SERIALIZER = 'pickle', #json
CELERY_TASK_SERIALIZER = 'pickle',
CELERY_IMPORTS=["vk_wall.tasks"]
UPD2:我的celery.py文件(位於firstapp / firstapp,與Django的settings.py處於同一級別)
from __future__ import absolute_import
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'firstapp.settings')
from django.conf import settings # noqa
app = Celery('firstapp')
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
@app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
UPD3:完整追溯
graph@engine:/var/www/engine$ celery worker -A firstapp.settings -l debug
Traceback (most recent call last):
File "/usr/local/bin/celery", line 11, in <module>
sys.exit(main())
File "/usr
/local/lib/python3.4/dist-packages/celery/__main__.py", line 30, in main
main()
File "/usr/local/lib/python3.4/dist-packages/celery/bin/celery.py", line 81, in main
cmd.execute_from_commandline(argv)
File "/usr/local/lib/python3.4/dist-packages/celery/bin/celery.py", line 770, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 309, in execute_from_commandline
argv = self.setup_app_from_commandline(argv)
File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 469, in setup_app_from_commandline
self.app = self.find_app(app)
File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 489, in find_app
return find_app(app, symbol_by_name=self.symbol_by_name)
File "/usr/local/lib/python3.4/dist-packages/celery/app/utils.py", line 229, in find_app
sym = symbol_by_name(app, imp=imp)
File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 492, in symbol_by_name
return symbol_by_name(name, imp=imp)
File "/usr/local/lib/python3.4/dist-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/usr/lib/python3.4/importlib/__init__.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2224, in _find_and_load_unlocked
ImportError: No module named 'firstapp'
在命令行中使用明確的設置路徑調用您的celery worker:
celery worker -A firstapp.celery
看來根本原因是:
vk_wall.tasks.add
發生這種情況是因為您沒有為芹菜指定設置來源。 它應該加載Django項目的設置或單獨的設置文件。
固定! 正確的選擇是:
芹菜工人-A發動機-l調試
另外,如果在settings.py中我將行更改為
CELERY_IMPORTS=["vk_wall.tasks"]
並在celery.py(引擎而不是firstapp)中:
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'engine.settings')
from django.conf import settings # noqa
app = Celery('engine')
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.