How to understand Tornado gen.coroutine

Question

I'm just a newbie to Tornado but not Python.

I'm trying to write an async client for CouchDB(using couchdb package).

After 1 day research/Googling, I found every posts just simply using HttpAsyncClient for example and not telling why and how gen.coroutine works. The source code just too complicated for me understand cause decorator after decorator HttpAsyncClient is a bad example to me...

Answer 1

Frankly, for ma it was only after reading source, that I (partially) understood the logic

It sort of works like this if you have decorated somefunction() with tornado.gen :

1. in statement yield somefunction() somefunction() is actually called
2. Since it's a wrapper, not your code is executed but tornado.coroutine.gen . It runs your code until the first yield in somefunction
3. If a Future(placehodler) is yielded(back to decorator code!), tornado says: "OK, when this future resolves, schedule a task(IOLoop callback) for IOLoop so that it calls some another_callback() ".
4. To track how far the execution of your somefunction() is gone, Tornado maintains a special object called Runner . It remembers the last yield statement which blocked the execution of your somefunction() and is first run when decorator gen is executed.
5. After point 3., where Future was "registered", this Runner returns from its main run() method and decorator exits, returning Future of its own
6. When Future from point 3. is ready, it adds a task to IOLoop, which then calls another_callback . Latter is a special callback created by Tornado, shortly put, it run() s the same Runner as was running when in point 3. a newly resolved Future was yielded.

7. Runner uses .send() method to inject a value of newly resolved Future back to your somefuncion , which cause ait to be assigned to a variable (if any) in your function in statement like this:

    a = yield some_other_coroutine_for_example_async_client_fetch() 

OK, that's a gist, there are lots of deatails, some of which I canot wrap my head around, especially, exception handling, but HTH

Answer 2

The first answer covers a lot. I just want to let u know a simpler abstraction. @Gen.coroutine

res = yield foo()

The coroutine idea is asyncly execute foo() , especially when foo takes a lot IO or network job. The yield could fire up the foo() execution and transfer control out , say, to caller. And it leaves a Runner to register this foo() task as a future obj. Then when foo() successfully return a result, this is magic happen, the Runner will send the result back, in the form of yield statement execution result (tell the diff between value yield and yield statement execution result).