![](/img/trans.png)
[英]RuntimeError: asyncio.run() cannot be called from a running event loop
[英]"asyncio.run() cannot be called from a running event loop" when using Jupyter Notebook
我想使用 asyncio 获取网页 html。
我在 jupyter notebook 中运行以下代码:
import aiofiles
import aiohttp
from aiohttp import ClientSession
async def get_info(url, session):
resp = await session.request(method="GET", url=url)
resp.raise_for_status()
html = await resp.text(encoding='GB18030')
with open('test_asyncio.html', 'w', encoding='utf-8-sig') as f:
f.write(html)
return html
async def main(urls):
async with ClientSession() as session:
tasks = [get_info(url, session) for url in urls]
return await asyncio.gather(*tasks)
if __name__ == "__main__":
url = ['http://huanyuntianxiazh.fang.com/house/1010123799/housedetail.htm', 'http://zhaoshangyonghefu010.fang.com/house/1010126863/housedetail.htm']
result = asyncio.run(main(url))
但是,它返回RuntimeError: asyncio.run() cannot be called from a running event loop
问题是什么?
如何解决?
asyncio.run()
文档说:
当另一个异步事件循环在同一线程中运行时, 无法调用此函数。
在您的情况下, jupyter ( IPython ≥ 7.0 )已经在运行事件循环:
您现在可以在 IPython 终端和笔记本的顶层使用 async/await,它应该——在大多数情况下——“正常工作”。 将 IPython 更新到 7+ 版,将 IPykernel 更新到 5+ 版,然后你就可以参加比赛了。
因此,您不需要自己启动事件循环,而是可以直接调用await main(url)
,即使您的代码位于任何异步函数之外。
Jupyter / IPython
async def main():
print(1)
await main()
Python (≥ 3.7) 或更早版本的 IPython
import asyncio
async def main():
print(1)
asyncio.run(main())
在您的代码中,将给出:
url = ['url1', 'url2']
result = await main(url)
for text in result:
pass # text contains your html (text) response
警告
与 IPython 相比,Jupyter 使用循环的方式略有不同。
添加到cglacet
的答案 - 如果想检测循环是否正在运行并自动调整(即在现有循环上运行main()
,否则asyncio.run()
),这里有一个可能被证明有用的片段:
# async def main():
# ...
try:
loop = asyncio.get_running_loop()
except RuntimeError: # 'RuntimeError: There is no current event loop...'
loop = None
if loop and loop.is_running():
print('Async event loop already running. Adding coroutine to the event loop.')
tsk = loop.create_task(main())
# ^-- https://docs.python.org/3/library/asyncio-task.html#task-object
# Optionally, a callback function can be executed when the coroutine completes
tsk.add_done_callback(
lambda t: print(f'Task done with result={t.result()} << return val of main()'))
else:
print('Starting new event loop')
asyncio.run(main())
结合 Pankaj Sharma 和 Jean Monet 的方法,我编写了以下代码片段,它充当 asyncio.run(语法略有不同),但也适用于 Jupyter 笔记本。
class RunThread(threading.Thread):
def __init__(self, func, args, kwargs):
self.func = func
self.args = args
self.kwargs = kwargs
super().__init__()
def run(self):
self.result = asyncio.run(self.func(*self.args, **self.kwargs))
def run_async(func, *args, **kwargs):
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop and loop.is_running():
thread = RunThread(func, args, kwargs)
thread.start()
thread.join()
return thread.result
else:
return asyncio.run(func(*args, **kwargs))
用法:
async def test(name):
await asyncio.sleep(5)
return f"hello {name}"
run_async(test, "user") # blocks for 5 seconds and returns "hello user"
我发现unsync
包对于编写在 Python 脚本和 Jupyter REPL 中行为相同的代码很有用。
import asyncio
from unsync import unsync
@unsync
async def demo_async_fn():
await asyncio.sleep(0.1)
return "done!"
print(demo_async_fn().result())
正如 cglacet 提到的那样,文档说
当另一个异步事件循环在同一线程中运行时,无法调用此函数。
您可以使用另一个线程,即 -
class ResolveThread(threading.Thread):
def __init__(self,result1,fun,url):
self.result1= result1
self.fun = fun
self.url = url
threading.Thread.__init__(self)
def run(self):
result1[0] = asyncio.run(self.fun(self.url))
result1 = [None]
sp = ResolveThread(result1)
sp.start()
sp.join() # connect main thread
result = result1[0]
Mark 稍微简化了解决方案:
import threading
class RunThread(threading.Thread):
def __init__(self, coro):
self.coro = coro
self.result = None
super().__init__()
def run(self):
self.result = asyncio.run(self.coro)
def run_async(coro):
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop and loop.is_running():
thread = RunThread(coro)
thread.start()
thread.join()
return thread.result
else:
return asyncio.run(coro)
像 async.run run_async()
async.run()
,即run_async(test("user"))
。
要添加到现有的答案,一个没有外部库的更短的版本,它允许在 jupyter 内部和外部运行并允许获取返回值:
try:
asyncio.get_running_loop()
# we need to create a separate thread so we can block before returning
with ThreadPoolExecutor(1) as pool:
result = pool.submit(lambda: asyncio.run(myfunc()))).result()
except RuntimeError:
# no event loop running
result = asyncio.run(myfunc())
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.