[英]Python asyncio how to give priority to one function during concurrent execution of its subfunctions
Here is a schematic version of my current setup:这是我当前设置的示意图:
import asyncio
async def child(args):
# await some things
async def parent(args):
# await some other things
if some_condition:
await asyncio.gather(child(args1), child(args2))
async def main():
await asyncio.gather(parent(args3), parent(args4))
asyncio.run(main())
So in essence I'm running the parent
function for different inputs concurrently.所以本质上,我同时为不同的输入运行
parent
function。 Every now and then some condition is met and as a result the corresponding parent
process starts the concurrent execution of the child
function for different inputs.时不时地满足一些条件,结果相应的
parent
进程开始并发执行child
进程 function 以获取不同的输入。 Since the child
function contains an await
, it gives up control to the event loop such that other functions within this event loop can run.由于
child
function 包含一个await
,它放弃对事件循环的控制,以便该事件循环中的其他函数可以运行。
My problem is that I only want the child
processes to give up control in favor of other child
processes of the same parent
process (ie essentially blocking the other parent
processes from running).我的问题是我只希望
child
进程放弃对同一parent
进程的其他child
进程的控制(即基本上阻止其他parent
进程运行)。 However, in practice this is not the case and other parent
processes are often running before the child
processes are finished.然而,实际上情况并非如此,其他
parent
进程通常在child
进程完成之前运行。 This increases the time it takes for the child
processes to complete (I actually have more than 2 parent
processes, I'm just limiting it here for simplicity).这增加了
child
进程完成所需的时间(我实际上有 2 个以上的parent
进程,为了简单起见,我只是在这里限制它)。
Is there a way to give priority to the child
processes, such that they are completed as fast as possible?有没有办法优先处理
child
进程,以便它们尽快完成? I tried a bunch of different asyncio
methods but nothing seems to work.我尝试了一堆不同的
asyncio
方法,但似乎没有任何效果。 Maybe there's an easy way to run the code inside the if statement in a synchronous way, such that it blocks the event loop (while still running the child
processes concurrently)?也许有一种简单的方法可以以同步方式在 if 语句中运行代码,从而阻止事件循环(同时仍同时运行
child
进程)?
Thanks in advance!提前致谢!
You can use an asyncio.Lock
.您可以使用
asyncio.Lock
。 With a lock you can simulate a shared resource between coroutines, locking the access to a coroutine if already other owns the lock.使用锁,您可以模拟协程之间的共享资源,如果其他人已经拥有该锁,则锁定对协程的访问。 In such way, can be implemented that one parent cannot invoke his childs if other parent is awaiting by the execution of his own childs.
以这种方式,可以实现一个父母不能调用他的孩子,如果另一个父母正在等待他自己的孩子的执行。
import asyncio
from time import time
# lock for parents when want to call childs
lock = asyncio.Lock()
async def child(parent, job):
print(f"child of {parent} doing {job} at {time()}")
await asyncio.sleep(3)
async def parent(name, jobs):
print(f"parent {name} doing stuff")
await asyncio.sleep(2)
async with lock: # only the childs of one parent will run concurrently
await asyncio.gather(child(name, jobs[0]), child(name, jobs[1]))
async def main():
await asyncio.gather(
parent("parent1", ["job1", "job2"]),
parent("parent2", ["job3", "job4"])
)
asyncio.get_event_loop().run_until_complete(main())
Output: Output:
parent parent1 doing stuff
parent parent2 doing stuff
child of parent1 doing job1 at 1650394577.7973263
child of parent1 doing job2 at 1650394577.7974186
child of parent2 doing job3 at 1650394580.801368
child of parent2 doing job4 at 1650394580.8014803
Note how the childs of parent2
started to do his jobs 3 seconds after the childs of parent1
ended.请注意 parent2 的孩子如何在
parent2
的parent1
结束后 3 秒开始做他的工作。
Without the lock you get something like this, every child starts at the same time.没有锁你会得到这样的东西,每个孩子同时开始。
parent parent1 doing stuff
parent parent2 doing stuff
child of parent1 doing job1 at 1650395040.6458762
child of parent1 doing job2 at 1650395040.6460035
child of parent2 doing job3 at 1650395040.6460583
child of parent2 doing job4 at 1650395040.6460981
Hope this answer your quetion.希望这能回答你的问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.