[英]Architecture Flask vs FastAPI
I have been tinkering around Flask and FastAPI to see how it acts as a server.我一直在修补 Flask 和 FastAPI 以了解它如何充当服务器。
One of the main things that I would like to know is how Flask and FastAPI deal with multiple requests from multiple clients.我想知道的主要事情之一是 Flask 和 FastAPI 如何处理来自多个客户端的多个请求。
Especially when the code has efficiency issues (long database query time).特别是当代码存在效率问题(数据库查询时间长)时。
So, I tried making a simple code to understand this problem.所以,我尝试编写一个简单的代码来理解这个问题。
The code is simple, when the client access the route, the application sleeps for 10 seconds before it returns results.代码很简单,当客户端访问路由时,应用程序在返回结果前会休眠 10 秒。
It looks something like this:它看起来像这样:
FastAPI快速API
import uvicorn
from fastapi import FastAPI
from time import sleep
app = FastAPI()
@app.get('/')
async def root():
print('Sleeping for 10')
sleep(10)
print('Awake')
return {'message': 'hello'}
if __name__ == "__main__":
uvicorn.run(app, host="127.0.0.1", port=8000)
Flask Flask
from flask import Flask
from flask_restful import Resource, Api
from time import sleep
app = Flask(__name__)
api = Api(app)
class Root(Resource):
def get(self):
print('Sleeping for 10')
sleep(10)
print('Awake')
return {'message': 'hello'}
api.add_resource(Root, '/')
if __name__ == "__main__":
app.run()
Once the applications are up, I tried accessing them at the same time through 2 different chrome clients.应用程序启动后,我尝试通过 2 个不同的 chrome 客户端同时访问它们。 The below are the results:以下是结果:
FastAPI快速API
Flask Flask
As you can see, for FastAPI, the code first waits 10 seconds before processing the next request.如您所见,对于 FastAPI,代码首先等待 10 秒,然后再处理下一个请求。 Whereas for Flask, the code processes the next request while the 10-second sleep is still happening.而对于 Flask,代码在 10 秒睡眠仍在发生时处理下一个请求。
Despite doing a bit of googling, there is not really a straight answer on this topic.尽管做了一些谷歌搜索,但这个话题并没有真正的直接答案。
If anyone has any comments that can shed some light on this, please drop them in the comments.如果有人有任何评论可以阐明这一点,请将它们放在评论中。
Your opinions are all appreciated.您的意见都是值得赞赏的。 Thank you all very much for your time.非常感谢大家的时间。
EDIT An update on this, I am exploring a bit more and found this concept of Process manager.编辑对此的更新,我正在探索更多并发现了流程管理器的这个概念。 For example, we can run uvicorn using a process manager (gunicorn).例如,我们可以使用进程管理器(gunicorn)运行 uvicorn。 By adding more workers, I am able to achieve something like Flask.通过添加更多的工人,我能够实现像 Flask 这样的东西。 Still testing the limits of this, however.然而,仍在测试这个极限。 https://www.uvicorn.org/deployment/ https://www.uvicorn.org/deployment/
Thanks to everyone who left comments.感谢所有留下评论的人。 Appreciate it.欣赏它。
This seemed a little interesting, so i ran a little tests with ApacheBench
:这似乎有点有趣,所以我用ApacheBench
进行了一些测试:
Flask Flask
from flask import Flask
from flask_restful import Resource, Api
app = Flask(__name__)
api = Api(app)
class Root(Resource):
def get(self):
return {"message": "hello"}
api.add_resource(Root, "/")
FastAPI快速API
from fastapi import FastAPI
app = FastAPI(debug=False)
@app.get("/")
async def root():
return {"message": "hello"}
I ran 2 tests for FastAPI, there was a huge difference:我对 FastAPI 进行了 2 次测试,结果有很大不同:
gunicorn -w 4 -k uvicorn.workers.UvicornWorker fast_api:app
uvicorn fast_api:app --reload
So here is the benchmarking results for 5000 requests with a concurrency of 500:所以这里是 5000 请求并发 500 的基准测试结果:
FastAPI with Uvicorn Workers带有 Uvicorn Workers 的 FastAPI
Concurrency Level: 500
Time taken for tests: 0.577 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 720000 bytes
HTML transferred: 95000 bytes
Requests per second: 8665.48 [#/sec] (mean)
Time per request: 57.700 [ms] (mean)
Time per request: 0.115 [ms] (mean, across all concurrent requests)
Transfer rate: 1218.58 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 6 4.5 6 30
Processing: 6 49 21.7 45 126
Waiting: 1 42 19.0 39 124
Total: 12 56 21.8 53 127
Percentage of the requests served within a certain time (ms)
50% 53
66% 64
75% 69
80% 73
90% 81
95% 98
98% 112
99% 116
100% 127 (longest request)
FastAPI - Pure Uvicorn FastAPI - 纯 Uvicorn
Concurrency Level: 500
Time taken for tests: 1.562 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 720000 bytes
HTML transferred: 95000 bytes
Requests per second: 3200.62 [#/sec] (mean)
Time per request: 156.220 [ms] (mean)
Time per request: 0.312 [ms] (mean, across all concurrent requests)
Transfer rate: 450.09 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 8 4.8 7 24
Processing: 26 144 13.1 143 195
Waiting: 2 132 13.1 130 181
Total: 26 152 12.6 150 203
Percentage of the requests served within a certain time (ms)
50% 150
66% 155
75% 158
80% 160
90% 166
95% 171
98% 195
99% 199
100% 203 (longest request)
For Flask :对于 Flask :
Concurrency Level: 500
Time taken for tests: 27.827 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 830000 bytes
HTML transferred: 105000 bytes
Requests per second: 179.68 [#/sec] (mean)
Time per request: 2782.653 [ms] (mean)
Time per request: 5.565 [ms] (mean, across all concurrent requests)
Transfer rate: 29.13 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 87 293.2 0 3047
Processing: 14 1140 4131.5 136 26794
Waiting: 1 1140 4131.5 135 26794
Total: 14 1227 4359.9 136 27819
Percentage of the requests served within a certain time (ms)
50% 136
66% 148
75% 179
80% 198
90% 295
95% 7839
98% 14518
99% 27765
100% 27819 (longest request)
Flask : Time taken for tests: 27.827 seconds Flask :测试时间:27.827 秒
FastAPI - Uvicorn : Time taken for tests: 1.562 seconds FastAPI - Uvicorn :测试时间:1.562 秒
FastAPI - Uvicorn Workers : Time taken for tests: 0.577 seconds FastAPI - Uvicorn Workers :测试时间:0.577 秒
With Uvicorn Workers FastAPI is nearly 48x faster than Flask, which is very understandable.使用 Uvicorn Workers FastAPI 比 Flask 快了近48 倍,这是非常可以理解的。 ASGI vs WSGI , so i ran with 1 concurreny: ASGI vs WSGI ,所以我跑了1个并发:
FastAPI - UvicornWorkers : Time taken for tests: 1.615 seconds FastAPI - UvicornWorkers :测试时间:1.615 秒
FastAPI - Pure Uvicorn : Time taken for tests: 2.681 seconds FastAPI - Pure Uvicorn :测试时间:2.681 秒
Flask : Time taken for tests: 5.541 seconds Flask :测试时间:5.541 秒
Flask with Waitress Flask 带女服务员
Server Software: waitress
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /
Document Length: 21 bytes
Concurrency Level: 1000
Time taken for tests: 3.403 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 830000 bytes
HTML transferred: 105000 bytes
Requests per second: 1469.47 [#/sec] (mean)
Time per request: 680.516 [ms] (mean)
Time per request: 0.681 [ms] (mean, across all concurrent requests)
Transfer rate: 238.22 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 4 8.6 0 30
Processing: 31 607 156.3 659 754
Waiting: 1 607 156.3 658 753
Total: 31 611 148.4 660 754
Percentage of the requests served within a certain time (ms)
50% 660
66% 678
75% 685
80% 691
90% 702
95% 728
98% 743
99% 750
100% 754 (longest request)
Gunicorn with Uvicorn Workers Gunicorn 与 Uvicorn 工人
Server Software: uvicorn
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /
Document Length: 19 bytes
Concurrency Level: 1000
Time taken for tests: 0.634 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 720000 bytes
HTML transferred: 95000 bytes
Requests per second: 7891.28 [#/sec] (mean)
Time per request: 126.722 [ms] (mean)
Time per request: 0.127 [ms] (mean, across all concurrent requests)
Transfer rate: 1109.71 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 28 13.8 30 62
Processing: 18 89 35.6 86 203
Waiting: 1 75 33.3 70 171
Total: 20 118 34.4 116 243
Percentage of the requests served within a certain time (ms)
50% 116
66% 126
75% 133
80% 137
90% 161
95% 189
98% 217
99% 230
100% 243 (longest request)
Pure Uvicorn, but this time 4 workers uvicorn fastapi:app --workers 4
纯 Uvicorn,但这次是 4 个工人uvicorn fastapi:app --workers 4
Server Software: uvicorn
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /
Document Length: 19 bytes
Concurrency Level: 1000
Time taken for tests: 1.147 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 720000 bytes
HTML transferred: 95000 bytes
Requests per second: 4359.68 [#/sec] (mean)
Time per request: 229.375 [ms] (mean)
Time per request: 0.229 [ms] (mean, across all concurrent requests)
Transfer rate: 613.08 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 20 16.3 17 70
Processing: 17 190 96.8 171 501
Waiting: 3 173 93.0 151 448
Total: 51 210 96.4 184 533
Percentage of the requests served within a certain time (ms)
50% 184
66% 209
75% 241
80% 260
90% 324
95% 476
98% 504
99% 514
100% 533 (longest request)
You are using the time.sleep()
function, in a async
endpoint.您正在async
端点中使用 time.sleep time.sleep()
function。 time.sleep()
is blocking and should never be used in asynchronous code. time.sleep()
是阻塞的,永远不应该在异步代码中使用。 What you should be using is probably the asyncio.sleep()
function:您应该使用的可能是asyncio.sleep()
function:
import asyncio
import uvicorn
from fastapi import FastAPI
app = FastAPI()
@app.get('/')
async def root():
print('Sleeping for 10')
await asyncio.sleep(10)
print('Awake')
return {'message': 'hello'}
if __name__ == "__main__":
uvicorn.run(app, host="127.0.0.1", port=8000)
That way, each request will take ~10 sec to complete, but you will be able to server multiple requests concurrently.这样,每个请求将需要大约 10 秒才能完成,但您将能够同时处理多个请求。
In general, async frameworks offer replacements for all blocking functions inside the standard library (sleep functions, IO functions, etc.).通常,异步框架为标准库中的所有阻塞函数(睡眠函数、IO 函数等)提供替换。 You are meant to use those replacements when writing async code and (optionally) await
them.您应该在编写异步代码和(可选) await
它们时使用这些替换。
Some non-blocking frameworks and libraries such as gevent, do not offer replacements.一些非阻塞框架和库(例如 gevent)不提供替代品。 They instead monkey-patch functions in the standard library to make them non-blocking.他们代替标准库中的猴子补丁函数,使它们成为非阻塞的。 This is not the case, as far as I know, for the newer async frameworks and libraries though, because they are meant to allow the developer to use the async-await syntax.但据我所知,对于较新的异步框架和库,情况并非如此,因为它们旨在允许开发人员使用 async-await 语法。
I think you are blocking an event queue in FastAPI which is asynchronous framework whereas in Flask requests are probably run each in new thread.我认为您在 FastAPI 中阻塞了一个事件队列,它是异步框架,而在 Flask 中,请求可能每个都在新线程中运行。 Move all CPU bound tasks to separate processes or in your FastAPI example just sleep on event loop (do not use time.sleep here).将所有 CPU 绑定任务移动到单独的进程或在您的 FastAPI 示例中只是在事件循环上休眠(不要在此处使用 time.sleep)。 In FastAPI run IO bound tasks asynchronously在 FastAPI 中异步运行 IO 绑定任务
Blocking operations will stop your event loop running the tasks.阻塞操作将停止您的事件循环运行任务。 When you are calling the sleep()
function, all the tasks (requests) are waiting until it's finished, thus killing all the benefits of asynchronous code execution.当您调用sleep()
function 时,所有任务(请求)都在等待它完成,从而扼杀了异步代码执行的所有好处。
To understand why this code is wrong for comparison, we should better understand how asynchronous code works in Python and have some knowledge of GIL.要理解为什么这段代码比较错误,我们应该更好地了解异步代码在 Python 中是如何工作的,并且对 GIL 有一些了解。 Concurrency and async code are well explained in the docs of FastAPI.并发和异步代码在 FastAPI 的文档中有很好的解释。
@Asotos has described why your code is slow and yes, you should use coroutines for I/O operations since they block the event loop execution ( sleep()
is a blocking operation). @Asotos 描述了为什么您的代码很慢,是的,您应该将协程用于 I/O 操作,因为它们会阻塞事件循环执行( sleep()
是阻塞操作)。 It is reasonably suggested to use async functions so that the event loop is not blocked, but for now, not all libraries have async versions.合理地建议使用异步函数,这样事件循环就不会被阻塞,但目前,并非所有库都有异步版本。
async
functions and asyncio.sleep
没有async
函数和asyncio.sleep
优化In case you cannot use the async version of the library, you can simply define your route functions as simple def
functions, not async def
.如果你不能使用库的异步版本,你可以简单地将你的路由函数定义为简单的def
函数,而不是async def
。
If the route function is defined as synchronous ( def
), FastAPI will smartly call this function in an external thread pool, and the main thread with event loop will not be blocked, and your benchmarks will be much better without using await asyncio.sleep()
.如果路由 function 被定义为同步( def
),FastAPI 会在外部线程池中智能地调用这个 function,并且不会阻塞带有事件循环的主线程,并且您的基准测试会更好,而不使用await asyncio.sleep()
. Greatly explained in this section.本节详细解释。
from time import sleep
import uvicorn
from fastapi import FastAPI
app = FastAPI()
@app.get('/')
def root():
print('Sleeping for 10')
sleep(10)
print('Awake')
return {'message': 'hello'}
if __name__ == "__main__":
uvicorn.run(app, host="127.0.0.1", port=8000)
BTW, you won't gain a lot of benefits, if operations run in the thread pool are CPU bound (eg heavy calculations) because of GIL .顺便说一句,如果在线程池中运行的操作由于GIL而受 CPU 限制(例如繁重的计算),您将不会获得很多好处。 CPU-bound tasks must be run in separate processes. CPU 密集型任务必须在单独的进程中运行。
What actually need to do in FastAPI is actually using Background Tasks in real scenario which you may sending email or doing heavy query on database and something like that在 FastAPI 中实际需要做的实际上是在实际场景中使用后台任务,您可能会发送 email 或对数据库进行大量查询等
FastAPI快速API
from fastapi import FastAPI, BackgroundTasks
import time
app = FastAPI()
def sleep(msg):
time.sleep(10)
print(msg)
@app.get('/')
async def root(background_tasks: BackgroundTasks):
msg= 'Sleeping for 10'
background_tasks.add_task(sleep, msg)
print('Awake')
return {'message': 'hello'}
then try check benchmarks然后尝试检查基准
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.