[英]Using Threadpool in an Async method without run_in_executor of asyncio.get_event_loop()
Folllowing is my code, which runs a long IO operation from an Async method using Thread Pool from Concurrent.Futures Package
as follows:以下是我的代码,它使用
Thread Pool from Concurrent.Futures Package
异步方法运行长 IO 操作,如下所示:
# io_bound/threaded.py
import concurrent.futures as futures
import requests
import threading
import time
import asyncio
data = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]
def sleepy(n):
time.sleep(n//2)
return n*2
async def ExecuteSleep():
l = len(data)
results = []
# Submit needs explicit mapping of I/p and O/p
# Output may not be in the same Order
with futures.ThreadPoolExecutor(max_workers=l) as executor:
result_futures = {d:executor.submit(sleepy,d) for d in data}
results = {d:result_futures[d].result() for d in data}
return results
if __name__ == '__main__':
print("Starting ...")
t1 = time.time()
result = asyncio.run(ExecuteSleep())
print(result)
print("Finished ...")
t2 = time.time()
print(t2-t1)
Following is my question:以下是我的问题:
What could be the potential issue if I run the Threadpool directly without using the following asyncio
apis:如果我不使用以下
asyncio
api 直接运行 Threadpool,可能会出现什么潜在问题:
loop = asyncio.get_event_loop() loop.run_in_executor(...)
I have reviewed the docs, ran simple test cases to me this looks perfectly fine and it will run the IO operation in the Background using the Custom thread pool, as listed here , I surely can't use pure async await to receive the Output and have to manage calls using map or submit methods, beside that I don't see a negative here.我已经查看了文档,对我运行了简单的测试用例,这看起来非常好,它将使用自定义线程池在后台运行 IO 操作,如此处所列,我当然不能使用纯异步等待来接收 Output 和必须使用 map 或提交方法来管理呼叫,除此之外,我在这里没有看到负面影响。
Ideone link of my code https://ideone.com/lDVLFh我的代码https://ideone.com/lDVLFh的 Ideone 链接
What could be the potential issue if I run the Threadpool directly
如果我直接运行 Threadpool,可能会出现什么问题
There is no issue if you just submit stuff to your thread pool and never interact with it or wait for results.如果您只是将东西提交到您的线程池并且从不与它交互或等待结果,那么没有问题。 But your code does wait for results¹.
但是您的代码确实在等待结果¹。
The issue is that ExecuteSleep
is blocking .问题是
ExecuteSleep
正在阻塞。 Although it's defined as async def
, it is async in name only because it doesn't await anything.尽管它被定义为
async def
,但它只是名称上的异步,因为它不等待任何东西。 While it runs, no other asyncio coroutines can run, so it defeats the main benefit of asyncio, which is running multiple coroutines concurrently.在它运行时,没有其他 asyncio 协程可以运行,因此它破坏了 asyncio 的主要优点,即同时运行多个协程。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.