[英]How to manage a single aiohttp.ClientSession?
作为一个学习练习,我正在尝试修改aiohttp的快速启动示例以使用单个ClientSession获取多个URL(文档建议通常应为每个应用程序创建一个ClientSession)。
import aiohttp
import asyncio
async def fetch(session, url):
async with session.get(url) as response:
return await response.text()
async def main(url, session):
print(f"Starting '{url}'")
html = await fetch(session, url)
print(f"'{url}' done")
urls = (
"https://python.org",
"https://twitter.com",
"https://tumblr.com",
"https://example.com",
"https://github.com",
)
loop = asyncio.get_event_loop()
session = aiohttp.ClientSession()
loop.run_until_complete(asyncio.gather(
*(loop.create_task(main(url, session)) for url in urls)
))
# session.close() <- this doesn't make a difference
但是,在协程外创建ClientSession显然不是可行的方法:
➜ python 1_async.py 1_async.py:30: UserWarning: Creating a client session outside of coroutine is a very dangerous idea session = aiohttp.ClientSession() Creating a client session outside of coroutine client_session: Starting 'https://python.org' Starting 'https://twitter.com' Starting 'https://tumblr.com' Starting 'https://example.com' Starting 'https://github.com' 'https://twitter.com' done 'https://example.com' done 'https://github.com' done 'https://python.org' done 'https://tumblr.com' done 1_async.py:34: RuntimeWarning: coroutine 'ClientSession.close' was never awaited session.close() Unclosed client session client_session: Unclosed connector connections: ['[(, 15024.110107067)]', '[(, 15024.147785039)]', '[(, 15024.252375415)]', '[(, 15024.292646968)]', '[(, 15024.342368087)]', '[(, 15024.466971983)]', '[(, 15024.602057745)]', '[(, 15024.837045568)]'] connector:
FWIW,在我尝试上述改变之前,这是main
:
async def main(url):
async with aiohttp.ClientSession() as session:
print(f"Starting '{url}'")
html = await fetch(session, url)
print(f"'{url}' done")
这样做的正确方法是什么? 我想过将一个url列表传递给main,但是无法以非顺序的方式使它工作。
Creating a client session outside of coroutine is a very dangerous idea
,因为当你创建它时,它被绑定到当前循环。 如果你之后更改了运行循环,它将挂起。 但如果你仔细使用它,你可以忽略它。 相关文档
至于我,我只是忽略了这个警告。 但它也很容易克服它:
async def create_session():
return aiohttp.ClientSession()
session = asyncio.get_event_loop().run_until_complete(create_session())
此外,您不需要显式创建Task
对象,只需执行此coroutine函数:
loop.run_until_complete(asyncio.gather(
*(main(url, session) for url in urls)
))
最后,别忘了close
是一个协程。 您应该使用loop.run_until_complete(session.close())
来关闭session
。
顺便说一句,如果你想创建一个类似异步的循环,你可以参考我的另一个答案 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.