运行 Celery 任务锁定 Flask

Question

I am trying to organize project structure for Flask+Celery app. When i worked in single file all works fine. But when i distributed code in modules, calling test_task.apply_async() is locking flask. My project structure:

web_spider/
 app/
  __init__.py
  rest/
   __init__.py
   views/
    __init__.py
    test_view.py
   flask_app.py
  task_runner/
   __init__.py
   celery_app.py
   tasks.py
 requirements.txt

test_view.py

import flask
from app.task_runner.tasks import test_task

api_test_view = flask.Blueprint('api_test_view', __name__)

@api_test_view.route('/')
def test_view():
 test_task.apply_async() #lock there
 return 'Hello, World!'

flask_app.py

import flask
from app.rest.views.api_test_view import test_view

flask_app = flask.Flask(__name__)
flask_app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'
flask_app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0'
flask_app.register_blueprint(test_view)

if __name__ == '__main__':
flask_app.run(debug=True)

celery_app.py

from app.rest.flask_app import flask_app
import celery

celery_app = celery.Celery(flask_app.name, broker=flask_app.config['CELERY_BROKER_URL'])

celery_app.conf.update(flask_app.config)

tasks.py

from celery import shared_task

@shared_task
def test_task():
 return 1 + 1

Answer 1

Probably, you don't load celery_app and shared_task can't find the Celery app to work with. Add to your web_spider/app/__init__.py or to web_spider/app/task_runner/__init__.py:

from app.task_runner.celery_app import celery_app

__all__ = ('celery_app',)

It's documented at https://docs.celeryproject.org/en/stable/django/first-steps-with-django.html#django-first-steps search for shared_task .