[英]How to terminate loop.run_in_executor with ProcessPoolExecutor gracefully?
[英]Why a process used requests.get() via loop.run_in_executor() not run concurrently?
我用loop.run_in_executor()
调用requests.get()
但它运行没有同意,而我评论这行,让它只调用打印loop.run_in_executor()
脚本同时运行。
import asyncio
import requests
async def get(name):
loop=asyncio.get_event_loop()
for i in range(15):
loop.run_in_executor(None,requests.get,'https://www.google.com',
dict(proxies={'http':'socks5://127.0.0.1:1080','https':'socks5://127.0.0.1:1080'}))
loop.run_in_executor(None,print,f'{name} thread {i}th get')
async def main():
await get('A')
await get('B')
await get('C')
loop=asyncio.get_event_loop()
loop.run_until_complete(main())
脚本的输出是有序的,例如
A thread 0th get
A thread 1th get
...
A thread 14th get
B thread 0th get
B thread 1th get
...
B thread 14th get
C thread 0th get
C thread 1th get
...
但是,在注释第7行和第8行代码后,输出的顺序混乱:
import asyncio
import requests
async def get(name):
loop=asyncio.get_event_loop()
for i in range(15):
#loop.run_in_executor(None,requests.get,'https://www.google.com',
#dict(proxies={'http':'socks5://127.0.0.1:1080','https':'socks5://127.0.0.1:1080'}))
loop.run_in_executor(None,print,f'{name} thread {i}th get')
async def main():
await get('A')
await get('B')
await get('C')
loop=asyncio.get_event_loop()
loop.run_until_complete(main())
输出的最后几行就像
...
C thread 5th get
C thread 0th get
C thread 13th get
B thread 12th get
C thread 2th get
C thread 6th get
C thread 10th get
为什么原始版本在没有并发的情况下运行?
您误解了执行逻辑,这是另一个更简单的示例,但是逻辑是相同的,希望对您有所帮助。 我的英语不好,代码比我的表达更好。
import asyncio
import random
import time
def sleep_print(a):
time.sleep(random.randint(1, 300) / 100)
print(a)
async def get(name):
loop = asyncio.get_event_loop()
for i in range(4):
loop.run_in_executor(None, sleep_print, f"first sleep_print: {name} thread {i}th get\n")
loop.run_in_executor(None, sleep_print, f"second sleep_print: {name} thread {i}th get\n")
async def main():
await get("A")
loop = asyncio.get_event_loop()
start = time.time()
loop.run_until_complete(main())
print("cost:", time.time() - start)
我的输出:
cost: 0.0006608963012695312
second sleep_print: A thread 1th get
second sleep_print: A thread 3th get
first sleep_print: A thread 2th get
first sleep_print: A thread 1th get
first sleep_print: A thread 0th get
second sleep_print: A thread 2th get
first sleep_print: A thread 3th get
second sleep_print: A thread 0th get
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.