[英]Asyncio “fire and forget” tasks in a separate thread
I have a long-running synchronous Python program, where I'd like to run ~10 "fire and forget" tasks every second.我有一个长时间运行的同步 Python 程序,我想每秒运行约 10 个“即发即弃”任务。 These tasks hit a remote API and do not need to return any value.
这些任务命中远程 API 并且不需要返回任何值。 I tried this answer , but it requires too much CPU/memory to spawn and maintain all the separate threads so I've been looking into asyncio.
我试过这个答案,但它需要太多的 CPU/内存来生成和维护所有单独的线程,所以我一直在研究 asyncio。
This answer explained nicely how to run "fire and forget" using asyncio.这个答案很好地解释了如何使用 asyncio 运行“一劳永逸”。 However, it requires using
run_until_complete()
, which waits until all the asyncio tasks are done.但是,它需要使用
run_until_complete()
,它会等待所有异步任务完成。 My program is using sync Python so this doesn't work for me.我的程序正在使用同步 Python 所以这对我不起作用。 Ideally, the code should be as simple as this, where
log_remote
won't block the loop:理想情况下,代码应该像这样简单,其中
log_remote
不会阻塞循环:
while True:
latest_state, metrics = expensive_function(latest_state)
log_remote(metrics) # <-- this should be run as "fire and forget"
I'm on Python 3.7.我在 Python 3.7。 How can I easily run this using asyncio on another thread?
如何在另一个线程上使用 asyncio 轻松运行它?
You can start a single event loop in a single background thread and use it for all your fire&forget tasks.您可以在单个后台线程中启动单个事件循环,并将其用于您的所有 fire&forget 任务。 For example:
例如:
import asyncio, threading
_loop = None
def fire_and_forget(coro):
global _loop
if _loop is None:
_loop = asyncio.new_event_loop()
threading.Thread(target=_loop.run_forever, daemon=True).start()
_loop.call_soon_threadsafe(asyncio.create_task, coro)
With that in place, you can just call fire_and_forget
on a coroutine object, created by calling an async def
:有了它,您可以在通过调用
async def
创建的协程fire_and_forget
上调用 fire_and_forget :
# fire_and_forget defined as above
import time
async def long_task(msg):
print(msg)
await asyncio.sleep(1)
print('done', msg)
fire_and_forget(long_task('foo'))
fire_and_forget(long_task('bar'))
print('continuing with something else...')
time.sleep(3)
Note that log_remote
will need to actually be written as an async def
using asyncio, aiohttp instead of requests, etc.请注意,
log_remote
实际上需要使用 asyncio、aiohttp 而不是请求等编写为async def
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.