[英]Flask: Background thread sees a non-empty queue as empty
When I run a Flask app in uwsgi, the background thread and the app functions see different values when querying size of the same Queue. 当我在uwsgi中运行Flask应用程序时,后台线程和应用程序函数在查询同一队列的大小时会看到不同的值。
GET
call returns the queue size. GET
调用返回队列大小。 POST
call adds an element to the Queue. POST
调用会向Queue添加一个元素。 When the app is from the shell using python tester.py
, I get the expected result: 当应用程序来自使用
python tester.py
的shell时,我得到了预期的结果:
2014-06-07 14:20:50.677995 Queue size is: 0
127.0.0.1 - - [07/Jun/2014 14:20:51] "POST /addMessage/X HTTP/1.1" 200 -
2014-06-07 14:20:51.679277 Queue size is: 1
2014-06-07 14:20:52.680425 Queue size is: 1
2014-06-07 14:20:53.681566 Queue size is: 1
2014-06-07 14:20:54.682708 Queue size is: 1
127.0.0.1 - - [07/Jun/2014 14:20:55] "POST /addMessage/Y HTTP/1.1" 200 -
2014-06-07 14:20:55.687755 Queue size is: 2
2014-06-07 14:20:56.688867 Queue size is: 2
However, when the app is executed using uwsgi
, I get the following in the logs: 但是,当使用
uwsgi
执行应用程序时,我在日志中获得以下内容:
2014-06-07 14:17:42.056863 Queue size is: 0
2014-06-07 14:17:43.057952 Queue size is: 0
[pid: 9879|app: 0|req: 6/6] 127.0.0.1 () {24 vars in 280 bytes} [Sat Jun 7 14:17:43 2014] POST /addMessage/X => generated 16 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 71 bytes (1 switches on core 0)
2014-06-07 14:17:44.059037 Queue size is: 0
2014-06-07 14:17:45.060118 Queue size is: 0
[pid: 9879|app: 0|req: 7/7] 127.0.0.1 () {24 vars in 280 bytes} [Sat Jun 7 14:17:45 2014] POST /addMessage/X => generated 16 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 71 bytes (1 switches on core 0)
2014-06-07 14:17:46.061205 Queue size is: 0
2014-06-07 14:17:47.062286 Queue size is: 0
When running under uwsgi, the background thread does not see the same queue as the app. 在uwsgi下运行时,后台线程看不到与应用程序相同的队列。 Why is that?
这是为什么? How can I make these two threads look at the same Queue object?
如何让这两个线程看到同一个Queue对象?
Updates 更新
app.logger
), and I can only see print
s. app.logger
),我只能看到print
。 This means that the thread is running, but it can't do anything with app.logger
. app.logger
执行任何app.logger
。 .ini
configuration .ini
配置 [uwsgi]
http-socket = :9002
plugin = python
wsgi-file = /home/ubuntu/threadtest-uwsgi.py
enable-threads = true
workers = 1
chdir = /home/ubuntu/thread-tester/thread_tester
from flask import Flask, jsonify
import Queue
from threading import Thread
import time
import datetime
import logging
import sys
logging.basicConfig(stream=sys.stderr,
format='%(asctime)s %(levelname)s - %(message)s')
app = Flask(__name__)
messages = Queue.Queue()
def print_queue_size():
while True:
app.logger.debug("%s Queue size is: %d" % (datetime.datetime.now(),
messages.qsize()))
time.sleep(1)
t = Thread(target=print_queue_size, args=())
t.setDaemon(True)
t.start()
@app.route("/queueSize", methods=["GET"])
def get_queue_size():
return jsonify({"qsize": messages.qsize()}), 200
@app.route("/addMessage/<message>", methods=["POST"])
def add_message_to_queue(message):
messages.put(message)
return jsonify({"qsize": messages.qsize()}), 200
if __name__ == "__main__":
app.run(port=6000)
From the Things to Know documenation page : 从事物知识文档页面 :
uWSGI tries to (ab)use the Copy On Write semantics of the
fork()
call whenever possible.uWSGI尽可能尝试(ab)使用
fork()
调用的Copy On Write语义。 By default it will fork after having loaded your applications to share as much of their memory as possible.默认情况下,它会在加载应用程序之后进行分叉,以尽可能多地共享内存。 If this behavior is undesirable for some reason, use the
lazy
option.如果由于某种原因这种行为是不合需要的,请使用
lazy
选项。 This will instruct uWSGI to load the applications after each worker's fork().这将指示uWSGI在每个worker的fork()之后加载应用程序。 Lazy mode changes the way graceful reloading works: instead of reloading the whole instance, each worker is reloaded in chain.
延迟模式改变了正常重新加载的工作方式:不是重新加载整个实例,而是在链中重新加载每个工作程序。 If you want “lazy app loading”, but want to maintain the standard uWSGI reloading behaviour, starting from 1.3 you can use the
lazy-apps
option.如果你想要“延迟应用程序加载”,但想要保持标准的uWSGI重新加载行为,从1.3开始你可以使用
lazy-apps
选项。
Your Flask app is started when uWSGI starts, then the one worker process is forked . 您的Flask应用程序在uWSGI启动时启动,然后分叉一个工作进程。 On forking, the
Queue
object is empty, and no longer shared with the original process. 在分叉时,
Queue
对象为空,不再与原始进程共享。 The thread isn't taken along. 线程没有被带走。
Try setting the lazy-apps
option to delay the loading of the Flask app until the worker is started. 尝试设置
lazy-apps
选项以延迟加载Flask应用程序,直到启动工作程序。
The documentation link in @Martijn Pieters' answer notes that lazy-apps
may consume more memory than preforking. @Martijn Pieters答案中的文档链接指出,
lazy-apps
可能比预先消耗更多的内存。 If you're concerned about this, you may also wish to consider the @postfork
decorator to have more granular control over what gets run after forking. 如果您对此感到担心,您可能还希望考虑
@postfork
装饰器对分叉后运行的内容进行更精细的控制。 You could create your Queue inside a @postfork
-decorated function and it will get created in each worker. 您可以在
@postfork
-decorated函数中创建您的Queue,它将在每个worker中创建。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.