[英]python - how to implement a C-function as awaitable (coroutine)
Environment: cooperative RTOS in C and micropython virtual machine is one of the tasks. 环境:C和micropython虚拟机中的协作RTOS是其中的任务之一。
To make the VM not block the other RTOS tasks, I insert RTOS_sleep()
in vm.c:DISPATCH()
so that after every bytecode is executed, the VM relinquishes control to the next RTOS task. 为了使VM不阻止其他RTOS任务,我在
vm.c:DISPATCH()
插入RTOS_sleep()
,以便在执行每个字节码后,VM放弃对下一个RTOS任务的控制。
I created a uPy interface to asynchronously obtain data from a physical data bus - could be CAN, SPI, ethernet - using producer-consumer design pattern. 我创建了一个uPy接口,使用生产者 - 消费者设计模式从物理数据总线异步获取数据 - 可以是CAN,SPI,以太网。
Usage in uPy: 在uPy中的用法:
can_q = CANbus.queue()
message = can_q.get()
The implementation in C is such that can_q.get()
does NOT block the RTOS: it polls a C-queue and if message is not received, it calls RTOS_sleep()
to give another task the chance to fill the queue. C中的实现是这样的,
can_q.get()
不阻止RTOS:它轮询C队列,如果没有收到消息,它调用RTOS_sleep()
给另一个任务机会填充队列。 Things are synchronized because the C-queue is only updated by another RTOS task and RTOS tasks only switch when RTOS_sleep()
is called ie cooperative 事情是同步的,因为C队列仅由另一个RTOS任务更新,而RTOS任务仅在
RTOS_sleep()
时切换,即合作
The C-implementation is basically: C实现基本上是:
// gives chance for c-queue to be filled by other RTOS task
while(c_queue_empty() == true) RTOS_sleep();
return c_queue_get_message();
Although the Python statement can_q.get()
does not block the RTOS, it does block the uPy script. 虽然Python语句
can_q.get()
不会阻止RTOS,但它会阻止uPy脚本。 I'd like to rewrite it so I can use it with async def
ie coroutine and have it not block the uPy script. 我想重写它,所以我可以使用它与
async def
即协同程序 ,并让它不阻止uPy脚本。
Not sure of the syntax but something like this: 不确定语法,但这样的事情:
can_q = CANbus.queue()
message = await can_q.get()
QUESTION 题
How do I write a C-function so I can await
on it? 如何编写C函数以便我可以
await
它?
I would prefer a CPython and micropython answer but I would accept a CPython-only answer. 我更喜欢CPython和micropython的答案,但我会接受仅限CPython的答案。
Note: this answer covers CPython and the asyncio framework. 注意:这个答案涵盖了CPython和asyncio框架。 The concepts, however, should apply to other Python implementations as well as other async frameworks.
但是,这些概念应该适用于其他Python实现以及其他异步框架。
How do I write a C-function so I can
await
on it?如何编写C函数以便我可以
await
它?
The simplest way to write a C function whose result can be awaited is by having it return an already made awaitable object, such as an asyncio.Future
. 编写可以等待结果的C函数的最简单方法是让它返回一个已经等待的对象,例如
asyncio.Future
。 Before returning the Future
, the code must arrange for the future's result to be set by some asynchronous mechanism. 在返回
Future
之前,代码必须安排将来的结果由一些异步机制设置。 All of these coroutine-based approaches assume that your program is running under some event loop that knows how to schedule the coroutines. 所有这些基于协程的方法都假定您的程序在一些知道如何安排协同程序的事件循环下运行。
But returning a future isn't always enough - maybe we'd like to define an object with an arbitrary number of suspension points. 但是回归未来并不总是足够 - 也许我们想要定义一个具有任意数量的悬挂点的对象。 Returning a future suspends only once (if the returned future is not complete), resumes once the future is completed, and that's it.
返回一个未来只暂停一次(如果返回的未来未完成),一旦未来完成就恢复,就是这样。 An awaitable object equivalent to an
async def
that contains more than one await
cannot be implemented by returning a future, it has to implement a protocol that coroutines normally implement. await
包含多个await
的async def
await
无法通过返回未来来实现,它必须实现协同程序通常实现的协议。 This is somewhat like an iterator implementing a custom __next__
and be used instead of a generator. 这有点像实现自定义
__next__
的迭代器,而不是使用生成器。
To define our own awaitable type, we can turn to PEP 492, which specifies exactly which objects can be passed to await
. 为了定义我们自己的等待类型,我们可以转向PEP 492,它精确地指定了哪些对象可以传递给
await
。 Other than Python functions defined with async def
, user-defined types can make objects awaitable by defining the __await__
special method, which Python/C maps to the tp_as_async.am_await
part of the PyTypeObject
struct. 比定义Python函数其它
async def
,用户定义的类型可以使物体awaitable通过定义__await__
特殊方法,它的Python / C映射到tp_as_async.am_await
所述的一部分PyTypeObject
结构。
What this means is that in Python/C, you must do the following: 这意味着在Python / C中,您必须执行以下操作:
tp_as_async
field of your extension type. tp_as_async
字段指定非NULL值。 am_await
member point to a C function that accepts an instance of your type and returns an instance of another extension type that implements the iterator protocol , ie defines tp_iter
(trivially defined as PyIter_Self
) and tp_iternext
. am_await
成员指向一个接受你的类型实例的C函数,并返回另一个实现迭代器协议的扩展类型的实例,即定义tp_iter
( PyIter_Self
定义为PyIter_Self
)和tp_iternext
。 tp_iternext
must advance the coroutine's state machine. tp_iternext
必须提升协程的状态机。 Each non-exceptional return from tp_iternext
corresponds to a suspension, and the final StopIteration
exception signifies the final return from the coroutine. tp_iternext
每个非异常返回对应于一个暂停,最后的StopIteration
异常表示来自协同程序的最终返回。 The return value is stored in the value
property of StopIteration
. StopIteration
的value
属性中。 For the coroutine to be useful, it must also be able to communicate with the event loop that drives it, so that it can specify when it is to be resumed after it has suspended. 为了使协程有用,它还必须能够与驱动它的事件循环通信,以便它可以指定何时在它被挂起后恢复。 Most of coroutines defined by asyncio expect to be running under the asyncio event loop, and internally use
asyncio.get_event_loop()
(and/or accept an explicit loop
argument) to obtain its services. asyncio定义的大多数协同程序都希望在asyncio事件循环下运行,并在内部使用
asyncio.get_event_loop()
(和/或接受显式loop
参数)来获取其服务。
To illustrate what the Python/C code needs to implement, let's consider simple coroutine expressed as a Python async def
, such as this equivalent of asyncio.sleep()
: 为了说明Python / C代码需要实现什么,让我们考虑表示为Python
async def
简单协程,例如asyncio.sleep()
等效:
async def my_sleep(n):
loop = asyncio.get_event_loop()
future = loop.create_future()
loop.call_later(n, future.set_result, None)
await future
# we get back here after the timeout has elapsed, and
# immediately return
my_sleep
creates a Future
, arranges for it to complete (its result to become set) in n seconds, and suspends itself until the future completes. my_sleep
创建一个Future
,安排它在n秒内完成(其结果设置),并暂停直到将来完成。 The last part uses await
, where await x
means "allow x
to decide whether we will now suspend or keep executing". 最后一部分使用
await
,其中await x
表示“允许x
决定我们现在是暂停还是继续执行”。 An incomplete future always decides to suspend, and the asyncio Task
coroutine driver special-cases yielded futures to suspend them indefinitely and connects their completion to resuming the task. 一个不完整的未来总是决定暂停,并且asyncio
Task
协程驱动程序特殊情况产生的期货无限期地暂停它们并连接它们的完成以恢复任务。 Suspension mechanisms of other event loops (curio etc) can differ in details, but the underlying idea is the same: await
is an optional suspension of execution. 其他事件循环(curio等)的暂停机制可能在细节上有所不同,但基本思想是相同的:
await
是可选的执行暂停。
__await__()
that returns a generator __await__()
返回一个生成器 To translate this to C, we have to get rid of the magic async def
function definition, as well as of the await
suspension point. 要将其转换为C,我们必须摆脱神奇的
async def
函数定义,以及await
暂停点。 Removing the async def
is fairly simple: the equivalent ordinary function simply needs to return an object that implements __await__
: 删除
async def
非常简单:等效的普通函数只需要返回一个实现__await__
的对象:
def my_sleep(n):
return _MySleep(n)
class _MySleep:
def __init__(self, n):
self.n = n
def __await__(self):
return _MySleepIter(self.n)
The __await__
method of the _MySleep
object returned by my_sleep()
will be automatically called by the await
operator to convert an awaitable object (anything passed to await
) to an iterator. 该
__await__
的方法_MySleep
通过返回的对象my_sleep()
将自动被调用await
运算符将awaitable对象(东西传递给转换await
),以迭代器。 This iterator will be used to ask the awaited object whether it chooses to suspend or to provide a value. 此迭代器将用于询问等待的对象是选择暂停还是提供值。 This is much like how the
for o in x
statement calls x.__iter__()
to convert the iterable x
to a concrete iterator . 这很像是
for o in x
语句中的for o in x
如何调用x.__iter__()
将iterable x
转换为具体的迭代器 。
When the returned iterator chooses to suspend, it simply needs to produce a value. 当返回的迭代器选择暂停时,它只需要生成一个值。 The meaning of the value, if any, will be interpreted by the coroutine driver, typically part of an event loop.
值的含义(如果有的话)将由协程驱动程序解释,通常是事件循环的一部分。 When the iterator chooses to stop executing and return from
await
, it needs to stop iterating. 当迭代器选择停止执行并从
await
返回时,它需要停止迭代。 Using a generator as a convenience iterator implementation, _MySleepIter
would look like this: 使用生成器作为便利迭代器实现,
_MySleepIter
将如下所示:
def _MySleepIter(n):
loop = asyncio.get_event_loop()
future = loop.create_future()
loop.call_later(n, future.set_result, None)
# yield from future.__await__()
for x in future.__await__():
yield x
As await x
maps to yield from x.__await__()
, our generator must exhaust the iterator returned by future.__await__()
. 当
await x
maps yield from x.__await__()
,我们的生成器必须耗尽future.__await__()
返回的迭代器future.__await__()
。 The iterator returned by Future.__await__
will yield if the future is incomplete, and return the future's result (which we here ignore, but yield from
actually provides) otherwise. Future.__await__
返回的迭代器将在未来不完整的情况下产生,并返回未来的结果(我们在这里忽略,但实际提供的yield from
)否则。
__await__()
that returns a custom iterator __await__()
返回自定义迭代器 The final obstacle for a C implementation of my_sleep
in C is the use of generator for _MySleepIter
. 在C中实现
my_sleep
的C的最后一个障碍是为_MySleepIter
使用生成器。 Fortunately, any generator can be translated to a stateful iterator whose __next__
executes the piece of code up to the next await or return. 幸运的是,任何生成器都可以转换为有状态迭代器,其
__next__
执行代码段直到下一个等待或返回。 __next__
implements a state machine version of the generator code, where yield
is expressed by returning a value, and return
by raising StopIteration
. __next__
实现生成器代码的状态机版本,其中yield
通过返回值表示,并通过提高StopIteration
return
。 For example: 例如:
class _MySleepIter:
def __init__(self, n):
self.n = n
self.state = 0
def __iter__(self): # an iterator has to define __iter__
return self
def __next__(self):
if self.state == 0:
loop = asyncio.get_event_loop()
self.future = loop.create_future()
loop.call_later(self.n, self.future.set_result, None)
self.state = 1
if self.state == 1:
if not self.future.done():
return next(iter(self.future))
self.state = 2
if self.state == 2:
raise StopIteration
raise AssertionError("invalid state")
The above is quite some typing, but it works, and only uses constructs that can be defined with native Python/C functions. 以上是一些打字,但它可以工作,并且只使用可以使用本机Python / C函数定义的构造。
Actually translating the two classes to C quite straightforward, but beyond the scope of this answer. 实际上将这两个类翻译成C非常简单,但超出了这个答案的范围。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.