[英]How to preserve Flask app context across Celery and SQLAlchemy
I'm building trying to learn Flask with a proof of concept Flask app, that takes a JSON payload, and uses SQLAlchemy to write it to a DB.我正在构建尝试通过概念证明 Flask 应用程序来学习 Flask,它采用 JSON 有效负载,并使用 SQLAlchemy 将其写入数据库。 I'm using celery to manage the write tasks.我正在使用芹菜来管理写入任务。
The app is structured该应用程序是结构化的
|-app.py
|-project
|-__init__.py
|-celery_utils.py
|-config.py
|-users
|-__init_.py
|-models.py
|-tasks.py
app.py
builds the flask app and celery instance. app.py
构建 flask 应用程序和 celery 实例。
app.py应用程序.py
from project import create_app, ext_celery
app = create_app()
celery = ext_celery.celery
@app.route("/")
def alive():
return "alive"
/project/__init__.py
is the application factory for the flask app. /project/__init__.py
是 flask 应用程序的应用程序工厂。 It instantiates the extensions, links everything together, and registers the blueprints.它实例化扩展,将所有内容链接在一起,并注册蓝图。
/project/ init .py /项目/初始化.py
import os
from flask import Flask
from flask_celeryext import FlaskCeleryExt
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy
from project.celery_utils import make_celery
from project.config import config
# instantiate extensions
db = SQLAlchemy()
migrate = Migrate()
ext_celery = FlaskCeleryExt(create_celery_app=make_celery)
def create_app(config_name=None):
if config_name is None:
config_name = os.environ.get("FLASK_CONFIG", "development")
# instantiate the app
app = Flask(__name__)
# set config
app.config.from_object(config[config_name])
# set up extensions
db.init_app(app)
migrate.init_app(app, db)
ext_celery.init_app(app)
# register blueprints
from project.users import users_blueprint
app.register_blueprint(users_blueprint)
# shell context for flask cli
@app.shell_context_processor
def ctx():
return {"app": app, "db": db}
return app
/project/celery_utils.py
manages the creation of the celery instances /project/celery_utils.py /project/celery_utils.py
管理 celery 实例的创建/project/celery_utils.py
from celery import current_app as current_celery_app
def make_celery(app):
celery = current_celery_app
celery.config_from_object(app.config, namespace="CELERY")
return celery
In the users dir, I'm trying to manage the creation of a basic user with celery task management.在用户目录中,我正在尝试使用 celery 任务管理来管理基本用户的创建。
'/project/users/ init .py` is where I create the blueprints and routes. '/project/users/ init .py` 是我创建蓝图和路线的地方。
/project/users/ init .py /项目/用户/初始化.py
from flask import Blueprint, request, jsonify
from .tasks import divide, post_to_db
users_blueprint = Blueprint("users", __name__, url_prefix="/users", template_folder="templates")
from . import models, tasks
@users_blueprint.route('/users', methods=['POST'])
def users():
request_data = request.get_json()
task = post_to_db.delay(request_data)
response = {"id": task.task_id,
"status": task.status,
}
return jsonify(response)
@users_blueprint.route('/responses', methods=['GET'])
def responses():
request_data = request.get_json()
result = AsyncResult(id=request_data['id'])
response = result.get()
return jsonify(response)
/project/users/models.py
is a simple User model - however, it does manage to successfully remain in the context of the flask app if created from the flask app cli. /project/users/models.py
是一个简单的用户模型——但是,如果从 flask app cli 创建,它确实成功地保留在 flask app 的上下文中。
/project/users/models.py /项目/用户/模型.py
from project import db
class User(db.Model):
"""model for the user object"""
__tablename__ = "users"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
username = db.Column(db.String(128), unique=True, nullable=False)
email = db.Column(db.String(128), unique=True, nullable=False)
def __init__(self, username, email, *args, **kwargs):
self.username = username
self.email = email
Finally, /project/users/tasks.py
is where I handle the celery tasks for this dir.最后, /project/users/tasks.py
是我处理此目录的 celery 任务的地方。
/project/users/tasks.py /项目/用户/tasks.py
from celery import shared_task
from .models import User
from project import db
@shared_task()
def post_to_db(payload):
print("made it here")
user = User(**payload)
db.session.add(user)
db.session.commit()
db.session.close()
return True
The modules work, but as soon as I wire it all up and hit the endpoint with a JSON payload, I get the error message:这些模块可以正常工作,但是当我将它们全部连接起来并使用 JSON 负载命中端点时,我收到了错误消息:
RuntimeError: No application found. Either work inside a view function or push an application context. ...
I have tried to preserve the app context in tasks.py by:我试图通过以下方式在 tasks.py 中保留应用程序上下文:
...
from project import db, ext_celery
@ext_celery.shared_task()
def post_to_db(payload):
...
...
from project import db, ext_celery
@ext_celery.task()
def post_to_db(payload):
...
These error with: TypeError: exceptions must derive from BaseException
这些错误是: TypeError: exceptions must derive from BaseException
I've tried pushing the app context我试过推送应用上下文
...
from project import db
from app import app
@shared_task()
def post_to_db(payload):
with app.app_context():
...
This also errors with: TypeError: exceptions must derive from BaseException
这也有以下错误: TypeError: exceptions must derive from BaseException
I've tried importing celery from the app itself我试过从应用程序本身导入芹菜
...
from project import db
from app import celery
@celery.task()
def post_to_db(payload):
...
This also errors with: TypeError: exceptions must derive from BaseException
这也有以下错误: TypeError: exceptions must derive from BaseException
Any suggestions gratefully received.非常感谢收到任何建议。 There's a final piece of the puzzle I'm missing, and it's very frustrating.我缺少最后一块拼图,这非常令人沮丧。
With thanks to snakecharmerb感谢snakecharmerb
I had to add ContextTask to the make_celery()
function in /project/celery_utils.py我必须将 ContextTask 添加到 /project/celery_utils.py 中的make_celery()
函数
from celery import current_app as current_celery_app
def make_celery(app):
celery = current_celery_app
celery.config_from_object(app.config, namespace="CELERY")
class ContextTask(celery.Task):
def __call__(self, *args, **kwargs):
with app.app_context():
return self.run(*args, **kwargs)
celery.Task = ContextTask
return celery
And then a few tweaks in /project/users/tasks.py然后在 /project/users/tasks.py 中进行一些调整
from celery import shared_task
from .models import User
from project import db
@shared_task()
def post_to_db(payload):
user = User(**payload)
db.session.add(user)
db.session.commit()
db.session.close()
return True
Now I can see the user in the database, and my message queue is progressing as expected.现在我可以在数据库中看到用户,并且我的消息队列正在按预期进行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.