[英]Ordered execution for a Python asyncio.Semaphore
I'm running a number of parallel executions with asyncio.Semaphore
over an ordered List
.我正在使用
asyncio.Semaphore
在有序List
运行许多并行执行。
async def send_cus(cus_id):
async with ClientSession(connector=TCPConnector(limit=0)) as session:
num_parallel = asyncio.Semaphore(20)
async def send_cust_2(cust_id):
async with num_parallel:
# do something ...
tasks = list(
send_cust_2(cust_id)
for cus_id in my_ordered_lst
)
await asyncio.gather(*tasks)
The point is that it executes in an unordered manner, regardless of the my_ordered_lst
order.关键是它以无序方式执行,无论
my_ordered_lst
顺序如何。 I understand it is because we cannot get guaranteed the execution order for threads or process-forks without syncing elements.我理解这是因为我们无法保证没有同步元素的线程或进程分支的执行顺序。
Is there a way to get an ordered execution throughout the List
, while still having a parallel execution?有没有办法在整个
List
获得有序执行,同时仍然具有并行执行? Like first send 1st.就像第一次发送第一次一样。 acquire the lock, the send then send, acquire the lock, and so on.
获取锁,发送然后发送,获取锁,依此类推。
Prior to Python 3.7, asyncio.gather()
started the tasks in an arbitrary order.在 Python 3.7 之前,
asyncio.gather()
以任意顺序启动任务。 For 3.7 it was rewritten for efficiency with the side effect that it now creates the tasks in the order specified.对于 3.7,它被重写以提高效率,但副作用是它现在按照指定的顺序创建任务。
In Python 3.6 and earlier you can work around the issue by spawning the tasks manually in the desired order, and then calling gather
to wait for them to finish:在 Python 3.6 及更早版本中,您可以通过按所需顺序手动生成任务,然后调用
gather
等待它们完成来解决此问题:
tasks = [asyncio.create_task(send_cust_2(cust_id))
for cust_id in my_ordered_lst]
await asyncio.gather(*tasks)
Note that this will just start the tasks in the specified order.请注意,这只会以指定的顺序启动任务。 How they proceed executing past the first
await
will depend on what the individual tasks do and the timings of their respective operations.它们如何在第一个
await
继续执行将取决于各个任务的作用以及它们各自操作的时间。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.