[英]How can I synchronize asyncio with other OS threads?
I have a program with one main thread where I spawn a second thread that uses asyncio.我有一个带有一个主线程的程序,我在其中生成了第二个使用 asyncio 的线程。 Are there any tools provided to synchronize these two threads?
是否提供了任何工具来同步这两个线程? If everything was asyncio, I could do it with its synchronization primitives, eg:
如果一切都是异步的,我可以用它的同步原语来做,例如:
import asyncio
async def taskA(lst, evt):
print(f'Appending 1')
lst.append(1)
evt.set()
async def taskB(lst, evt):
await evt.wait()
print('Retrieved:', lst.pop())
lst = []
evt = asyncio.Event()
asyncio.get_event_loop().run_until_complete(asyncio.gather(
taskA(lst, evt),
taskB(lst, evt),
))
However, this does not work with multiple threads.但是,这不适用于多线程。 If I just use a
threading.Event
then it will block the asyncio thread.如果我只使用一个
threading.Event
那么它会阻塞 asyncio 线程。 I figured out I could defer the wait to an executor:我想我可以将等待推迟到执行者:
import asyncio
import threading
def taskA(lst, evt):
print(f'Appending 1')
lst.append(1)
evt.set()
async def taskB(lst, evt):
asyncio.get_event_loop().run_in_executor(None, evt.wait)
print('Retrieved:', lst.pop())
def targetA(lst, evt):
taskA(lst, evt)
def targetB(lst, evt):
asyncio.set_event_loop(asyncio.new_event_loop())
asyncio.get_event_loop().run_until_complete(taskB(lst, evt))
lst = []
evt = threading.Event()
threadA = threading.Thread(target=targetA, args=(lst, evt))
threadB = threading.Thread(target=targetB, args=(lst, evt))
threadA.start()
threadB.start()
threadA.join()
threadB.join()
However, having an executor thread only to wait for a mutex seems unnatural.但是,让执行程序线程只等待互斥锁似乎不自然。 Is this the way this is supposed to be done?
这是应该这样做的方式吗? Or is there any other way to wait for synchronization between OS threads asynchronously?
或者有没有其他方法可以异步等待操作系统线程之间的同步?
A simple way to synchronize an asyncio coroutine with an event coming from another thread is to await an asyncio.Event
in taskB, and set it from taskA using loop.call_soon_threadsafe
.将 asyncio 协程与来自另一个线程的事件同步的一种简单方法是在 taskB 中等待
asyncio.Event
,并使用loop.call_soon_threadsafe
从 taskA 设置它。
To be able to pass values and exceptions between the two, you can use futures;为了能够在两者之间传递值和异常,您可以使用期货; however then you are inventing much of
run_in_executor
.但是,您正在发明很多
run_in_executor
。 If the only job of taskA is to take tasks off a queue, you might as well make a single-worker "pool" and use it as your worker thread.如果 taskA 的唯一工作是从队列中取出任务,您不妨创建一个单工“池”并将其用作您的工作线程。 Then you can use
run_in_executor
as intended:然后您可以按预期使用
run_in_executor
:
worker = concurrent.futures.ThreadPoolExecutor(max_workers=1)
async def taskB(lst):
loop = asyncio.get_event_loop()
# or result = await ..., if taskA has a useful return value
# This will also propagate exceptions raised by taskA
await loop.run_in_executor(worker, taskA, lst)
print('Retrieved:', lst.pop())
The semantics are the same as in your version with an explicit queue - the queue is still there, it's just inside the ThreadPoolExecutor
.语义与具有显式队列的版本相同 - 队列仍然存在,它只是在
ThreadPoolExecutor
内部。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.