Why does an async consumer called in __init__ in a Tornado RequestHandler behave differently from statically called?

Question

I'm trying to make an async server using Tornado with a unique queue for each handler. A job is placed into a queue when the end point is called. I have a consumer function which asynchronously 'consumes' jobs from the queue. However, the behavior of consumer tends to vary depending on whether I call it as self.consumer() or AsyncHandler.consumer() . My initial guess is that it is because of instance level locking but can't find evidence for it. I fire 4 post requests consecutively. Here are the 2 snippets with their outputs.

import tornado.web
from tornado import gen
from time import sleep, time
from tornado.queues import Queue
from concurrent.futures import ThreadPoolExecutor
from tornado.ioloop import IOLoop

class AsyncHandler(tornado.web.RequestHandler):

    JOB_QUEUE = Queue()
    EXECUTOR = ThreadPoolExecutor()

    def post(self):
        job = lambda: sleep(3) or print("{}:handler called".format(int(time())))
        self.JOB_QUEUE.put(job)
        self.set_status(200)
        self.finish()

    @staticmethod
    @gen.coroutine
    def consumer():
        while True:
            job = yield AsyncHandler.JOB_QUEUE.get()
            print("qsize : {}".format(AsyncHandler.JOB_QUEUE.qsize()))
            print(AsyncHandler.JOB_QUEUE)
            output = yield AsyncHandler.EXECUTOR.submit(job)
            AsyncHandler.JOB_QUEUE.task_done()


if __name__ == "__main__":
    AsyncHandler.consumer()
    APP = tornado.web.Application([(r"/test", AsyncHandler)])
    APP.listen(9000)
    IOLoop.current().start()

This gives the expected output:

qsize : 0
<Queue maxsize=0 tasks=1>
1508618429:handler called
qsize : 2
<Queue maxsize=0 queue=deque([<function...<lambda> at 0x7fbf8f741400>, <function... <lambda> at 0x7fbf8f760ea0>]) tasks=3>
1508618432:handler called
qsize : 1
<Queue maxsize=0 queue=deque([<function AsyncHandler.post.<locals>.<lambda> at 0x7fbf8f760ea0>]) tasks=2>
1508618435:handler called
qsize : 0
<Queue maxsize=0 tasks=1>
1508618438:handler called

output = yield AsyncHandler.EXECUTOR.submit(job) takes 3 seconds to return output and so the outputs arrive at delay of 3 seconds. Also we can see the queue build up in the meanwhile.

Now to the interesting piece of code:

import tornado.web
from tornado import gen
from time import sleep, time
from tornado.queues import Queue
from concurrent.futures import ThreadPoolExecutor
from tornado.ioloop import IOLoop

class AsyncHandler(tornado.web.RequestHandler):
    JOB_QUEUE = Queue()
    EXECUTOR = ThreadPoolExecutor()

    def __init__(self, application, request, **kwargs):
        super().__init__(application, request, **kwargs)
        self.consumer()

    def post(self):
        job = lambda: sleep(3) or print("{}:handler called".format(int(time())))
        self.JOB_QUEUE.put(job)
        self.set_status(200)
        self.finish()

    @staticmethod
    @gen.coroutine
    def consumer():
        while True:
            job = yield AsyncHandler.JOB_QUEUE.get()
            print("qsize : {}".format(AsyncHandler.JOB_QUEUE.qsize()))
            print(AsyncHandler.JOB_QUEUE)
            output = yield AsyncHandler.EXECUTOR.submit(job)
            AsyncHandler.JOB_QUEUE.task_done()


if __name__ == "__main__":
    APP = tornado.web.Application([(r"/test", AsyncHandler)])
    APP.listen(9000)
    IOLoop.current().start()

The output weirdly (and pleasantly) looks like:

qsize : 0
<Queue maxsize=0 tasks=1>
qsize : 0
<Queue maxsize=0 tasks=2>
qsize : 0
<Queue maxsize=0 tasks=3>
qsize : 0
<Queue maxsize=0 tasks=4>
1508619138:handler called
1508619138:handler called
1508619139:handler called
1508619139:handler called

Note that now we're calling consumer inside __init__ . We can see the tasks build up and execute in parallel (without a queue build up), completing almost simultaneously. It's as if output = yield AsyncHandler.EXECUTOR.submit(job) is not blocking on the future. Even after a lot of experimentation I'm unable to explain this behavior. I'd really appreciate some help.

Answer 1

The first application has only one running consumer because it's executed once. Each request "blocks" (only one at a time is the loop) the consumer, so the next one will be processed after the previous.

The latter app starts a new consumer loop with each request (since RequestHandler is created per req). So the first request does not "block" the next one, 'cause there is brand new while True with get and submit ...