[英]How to interrupt Tornado coroutine
Suppose I have two functions that work like this:假设我有两个像这样工作的函数:
@tornado.gen.coroutine
def f():
for i in range(4):
print("f", i)
yield tornado.gen.sleep(0.5)
@tornado.gen.coroutine
def g():
yield tornado.gen.sleep(1)
print("Let's raise RuntimeError")
raise RuntimeError
In general, function f
might contain endless loop and never return (eg it can process some queue).通常,函数
f
可能包含无限循环并且永不返回(例如,它可以处理某个队列)。
What I want to do is to be able to interrupt it, at any time it yields.我想要做的是能够在它产生的任何时候中断它。
The most obvious way doesn't work.最明显的方法不起作用。 Exception is only raised after function
f
exits (if it's endless, it obviously never happens).异常仅在函数
f
退出后引发(如果它是无穷无尽的,它显然永远不会发生)。
@tornado.gen.coroutine
def main():
try:
yield [f(), g()]
except Exception as e:
print("Caught", repr(e))
while True:
yield tornado.gen.sleep(10)
if __name__ == "__main__":
tornado.ioloop.IOLoop.instance().run_sync(main)
Output:输出:
f 0
f 1
Let's raise RuntimeError
f 2
f 3
Traceback (most recent call last):
File "/tmp/test/lib/python3.4/site-packages/tornado/gen.py", line 812, in run
yielded = self.gen.send(value)
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
<...>
File "test.py", line 16, in g
raise RuntimeError
RuntimeError
That is, exception is only raised when both of the coroutines return (both futures resolve).也就是说,只有当两个协程都返回(两个期货都解决)时才会引发异常。
This's partially solved by tornado.gen.WaitIterator
, but it's buggy ( unless I'm mistaken ).这部分由
tornado.gen.WaitIterator
解决,但它有问题(除非我弄错了)。 But that's not the point.但这不是重点。
It still doesn't solve the problem of interrupting existing coroutines.它仍然没有解决中断现有协程的问题。 Coroutine continues to run even though the function that started it exits.
即使启动它的函数退出,协程也会继续运行。
EDIT : it seems like coroutine cancellation is something not really supported in Tornado, unlike in Python's asyncio, where you can easily throw CancelledError
at every yield point.编辑:Tornado 似乎并不真正支持协程取消,这与 Python 的 asyncio 不同,在 Python 的 asyncio 中,您可以在每个屈服点轻松抛出
CancelledError
。
If you use WaitIterator according to the instructions , and use a toro.Event to signal between coroutines, it works as expected:如果您根据说明使用 WaitIterator ,并使用toro.Event在协程之间发出信号,它会按预期工作:
from datetime import timedelta
import tornado.gen
import tornado.ioloop
import toro
stop = toro.Event()
@tornado.gen.coroutine
def f():
for i in range(4):
print("f", i)
# wait raises Timeout if not set before the deadline.
try:
yield stop.wait(timedelta(seconds=0.5))
print("f done")
return
except toro.Timeout:
print("f continuing")
@tornado.gen.coroutine
def g():
yield tornado.gen.sleep(1)
print("Let's raise RuntimeError")
raise RuntimeError
@tornado.gen.coroutine
def main():
wait_iterator = tornado.gen.WaitIterator(f(), g())
while not wait_iterator.done():
try:
result = yield wait_iterator.next()
except Exception as e:
print("Error {} from {}".format(e, wait_iterator.current_future))
stop.set()
else:
print("Result {} received from {} at {}".format(
result, wait_iterator.current_future,
wait_iterator.current_index))
if __name__ == "__main__":
tornado.ioloop.IOLoop.instance().run_sync(main)
For now, pip install toro
to get the Event class.现在,
pip install toro
来获取 Event 类。 Tornado 4.2 will include Event, see the changelog . Tornado 4.2 将包含事件, 请参阅更改日志。
Since version 5, Tornado runs on asyncio
event loop .从版本 5 开始, Tornado 在
asyncio
事件循环上运行。
On Python 3, the
IOLoop
is always a wrapper around theasyncio
event loop, andasyncio.Future
andasyncio.Task
are used instead of their Tornado counterparts.在 Python 3 上,
IOLoop
始终是asyncio
事件循环的包装器,并且使用asyncio.Future
和asyncio.Task
代替它们的 Tornado 对应项。
Hence you can use asyncio
Task cancellation, ie asyncio.Task.cancel
.因此您可以使用
asyncio
任务取消,即asyncio.Task.cancel
。
Your example with a queue reading while-true loop, might look like this.您的队列读取 while-true 循环示例可能如下所示。
import logging
from asyncio import CancelledError
from tornado import ioloop, gen
async def read_off_a_queue():
while True:
try:
await gen.sleep(1)
except CancelledError:
logging.debug('Reader cancelled')
break
else:
logging.debug('Pretend a task is consumed')
async def do_some_work():
await gen.sleep(5)
logging.debug('do_some_work is raising')
raise RuntimeError
async def main():
logging.debug('Starting queue reader in background')
reader_task = gen.convert_yielded(read_off_a_queue())
try:
await do_some_work()
except RuntimeError:
logging.debug('do_some_work failed, cancelling reader')
reader_task.cancel()
# give the task a chance to clean up, in case it
# catches CancelledError and awaits something
try:
await reader_task
except CancelledError:
pass
if __name__ == '__main__':
logging.basicConfig(level='DEBUG')
ioloop.IOLoop.instance().run_sync(main)
If you run it, you should see:如果你运行它,你应该看到:
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:root:Starting queue reader in background
DEBUG:root:Pretend a task is consumed
DEBUG:root:Pretend a task is consumed
DEBUG:root:Pretend a task is consumed
DEBUG:root:Pretend a task is consumed
DEBUG:root:do_some_work is raising
DEBUG:root:do_some_work failed, cancelling reader
DEBUG:root:Reader cancelled
Warning : This is not a working solution.警告:这不是一个有效的解决方案。 Look at the commentary.
看评论。 Still if you're new (as myself), this example can show the logical flow.
尽管如此,如果您是新手(就像我自己),这个例子可以显示逻辑流程。 Thanks @nathaniel-j-smith and @wgh
谢谢@nathaniel-j-smith 和@wgh
What is the difference using something more primitive, like global variable for instance?使用更原始的东西有什么区别,比如全局变量?
import asyncio
event = asyncio.Event()
aflag = False
async def short():
while not aflag:
print('short repeat')
await asyncio.sleep(1)
print('short end')
async def long():
global aflag
print('LONG START')
await asyncio.sleep(3)
aflag = True
print('LONG END')
async def main():
await asyncio.gather(long(), short())
if __name__ == '__main__':
asyncio.run(main())
It is for asyncio , but I guess the idea stays the same.它用于asyncio ,但我想这个想法保持不变。 This is a semi-question (why Event would be better?).
这是一个半问题(为什么Event会更好?)。 Yet solution yields exact result author needs:
然而,解决方案产生了作者需要的确切结果:
LONG START
short repeat
short repeat
short repeat
LONG END
short end
UPDATE: this slides may be really helpful in understanding core of a problem.更新:这张幻灯片可能对理解问题的核心很有帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.