[英]Why asyncio's run_in_executor gives so little parallelization when making HTTP requests?
[英]Why asyncio's run_in_executor blocks tornado's get handler?
我想在龙卷风的异步GET请求处理程序中运行一个缓慢的阻止方法(实际上是从第三方库中)。 将该方法设为:
def blocking_method(uid):
print("slow method started: ", uid)
time.sleep(10)
print("slow method done: ", uid)
return "slow method ({}) result".format(uid)
此外,我更喜欢在asyncio的事件循环中运行龙卷风服务器:
if __name__ == '__main__':
tornado.platform.asyncio.AsyncIOMainLoop().install()
loop = asyncio.get_event_loop()
loop.run_until_complete(make_app())
loop.run_forever()
我知道@run_in_executor
装饰器,但是它不适合我,因为我使用asyncio。 要在异步协程中运行阻塞方法,我应该使用asyncio.get_event_loop()
run_in_executor
方法。 这是一个示例,如何从此答案中做到这一点 :
import asyncio
async def main():
loop = asyncio.get_event_loop()
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
future1 = loop.run_in_executor(executor, blocking_method, 1)
future2 = loop.run_in_executor(executor, blocking_method, 2)
response1 = await future1
response2 = await future2
print(response1)
print(response2)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
而且效果很好,这是先前脚本的输出:
slow method started: 1
slow method started: 2
slow method done: 2
slow method done: 1
slow method (1) result
slow method (2) result
但是,如果我在龙卷风的RequestHandler的async def get
方法中使用了非常相同的技术:
class AsyncHandler(tornado.web.RequestHandler):
async def get(self):
# simple counter to distinguish requests
self.application.counter += 1
in_msg = "Registered request #{}, working...".format(self.application.counter)
print(in_msg)
loop = asyncio.get_event_loop()
future = loop.run_in_executor(self.application.executor,
blocking_method,
self.application.counter)
result = await future
out_msg = "Request processed, result: {}".format(result)
print(out_msg)
self.write(out_msg)
它阻止了处理程序的方法。 换句话说,如果我在多个浏览器选项卡中打开http://localhost:8888/
(将其设置为两个),那么我希望两个请求并行运行,并显示以下输出:
Registered request #1, working...
slow method started: 1
Registered request #2, working...
slow method started: 2
slow method done: 1
Request processed, result: slow method (1) result
slow method done: 2
Request processed, result: slow method (2) result
但是请求因此被执行:
Registered request #1, working...
slow method started: 1
slow method done: 1
Request processed, result: slow method (1) result
Registered request #2, working...
slow method started: 2
slow method done: 2
Request processed, result: slow method (2) result
那么,我哪里错了? 我应该怎么做才能允许并行执行请求处理程序?
这是描述我的问题的完整脚本:
import asyncio
import concurrent.futures
import time
import tornado.web
import tornado.platform
def blocking_method(uid):
print("slow method started: ", uid)
time.sleep(10)
print("slow method done: ", uid)
return "slow method ({}) result".format(uid)
class AsyncHandler(tornado.web.RequestHandler):
async def get(self):
# simple counter to distinguish requests
self.application.counter += 1
in_msg = "Registered request #{}, working...".format(self.application.counter)
print(in_msg)
loop = asyncio.get_event_loop()
future = loop.run_in_executor(self.application.executor,
blocking_method,
self.application.counter)
result = await future
out_msg = "Request processed, result: {}".format(result)
print(out_msg)
self.write(out_msg)
async def make_app():
handlers = [(r"/", AsyncHandler)]
app = tornado.web.Application(handlers, debug=True)
app.executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
app.counter = 0
app.listen(8888)
if __name__ == '__main__':
tornado.platform.asyncio.AsyncIOMainLoop().install()
loop = asyncio.get_event_loop()
loop.run_until_complete(make_app())
loop.run_forever()
浏览器将识别出您正在尝试在两个不同的选项卡中加载同一页面,并将第二个请求延迟到第一个请求完成为止。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.