Tornado await for Future object to resolve

Question

I have a handler with the following code:

class HelloHandler(RequestHandler):

      routing_pattern = "/hello"

      async def get(self):

          url = 'some_url_here'
          request = httpclient.HTTPRequest(url=url, streaming_callback=self.on_chunk)
          result = await downloader.fetch(request)
          print(result)
          self.write("done")


      @gen.coroutine
      def on_chunk(self, chunk):
          self.write(chunk)
          yield self.flush()

this code calls an async def function defined like so:

async def fetch(request):
    future = Future()
    await _qin.put(request)
    return future

I was hoping that inside my handler, things will stop on the await downloader.fetch(request) line until I set a value on the returned future. right now that would never happen so things should have stoped there. However it seems that the future is not actually awaited. The print(result) line shows a "" and things just speed on past that line. What am I doing wrong ? How can i make the function stop there and actually wait for the future to complete ? A side question ... is what I am doing in the on_chunk method correct ? I would like to await for the flush to happen in there but streaming_callback does not take an async function.

Answer 1

I was hoping that inside my handler, things will stop on the await downloader.fetch(request) line until I set a value on the returned future.

You are right.

But there's a problem with your code. Keep these points in mind *:

1. A coroutine ( async def function, or gen.coroutine decorated function) automatically returns a Future .
2. Anything you return from a coroutine, it will be wrapped inside a Future.

In async def fetch(request) function, you are returning a Future object. But as per the rule 2 above, your Future object will be wrapped inside the coroutine's Future object.

So, the reason that await downloader.fetch is not pausing your function is because the Future which is automatically returned by the fetch coroutine is being resolved instantly.

Your code should work as expected by awaiting twice:

result_future = await downloader.fetch(request)
result = await result_future

---

Personally, I find the double await a little inconsistent with the general convention. So, what I would do is I'd make the fetch(request) a regular function, instead of a coroutine:

def fetch(request):
    future = Future()
    # instead of running _qin.put yourself, 
    # ask the ioloop to run it
    ioloop.IOLoop.current().add_callback(_qin.put, request)
    return future

You can just ignore this if you're comfortable with using double await , or if you've to make a lot of changes in your code.

---

_{*Disclosure: The linked article is from my own blog.}