Python asyncio future add_done_callback then cancel the future

Question

I have a problem to understand how asyncio works if I create a future future = asyncio.Future() then add add_done_callback(done_callback) and after that cancel the future future.cancel() the done_callback not suppose to get fired? I tried to use the loop.run_forever() but I end up with infinite loop.

I have a small example code:

_future_set = asyncio.Future()


def done_callback(f):
    if f.exception():
        _future_set.set_exception(f.exception)
    elif f.cancelled():
        _future_set.cancel()
    else:
        _future_set.set_result(None)

async def check_stats(future):
    while not future.cancelled():
        print("not done")
        continue
    loop.stop()

def set(future):
    if not _future_set.done():
        future.add_done_callback(done_callback)


loop = asyncio.new_event_loop()
future = loop.create_future()

asyncio.ensure_future(check_stats(future), loop=loop)

set(future)
future.cancel()
loop.run_forever()  # infinite
print(_future_set.cancelled())

I know something is missing and maybe this is not the behavior but I will be happy for a little help here.

I am using python 3.6

**update After set is fire and I bind the add_done_callback to the future when I cancel the future and the state of the future change to cancelled and done then I expect that _future_set will be cancelled too. and print(_future_set.cancelled()) will be True

Answer 1

From the docstring (on my unix system) help(loop.run_forever) :

  run_forever() method of asyncio.unix_events._UnixSelectorEventLoop instance
    Run until stop() is called.

When you call loop.run_forever() the program will not progress beyond that line until stop() is called on the ioloop instance, and there's nothing in your code doing so.

loop.run_forever() is essentially doing:

def run_forever(self):
    while not self.stopped:
        self.check_for_things_to_do()

Without knowing a little more as to what you're trying to achieve, it's hard to help you further. However it seems that you're expecting loop.run_forever() to be asynchronous in the execution of the python code, however this is not the case. The IOLoop will keep looping and check filevents and fire callbacks on futures, and will only return back to the point it's called if it is told to stop looping.

---

Ah, I realise now what you're expecting to happen. You need to register the futures with the ioloop, either by doing future = loop.create_future() or future = asyncio.Future(loop=loop) . The former is the preferred method for creating futures. NB the code will still run forever at the loop.run_forever() call unless it is stopped, so your print statement will still never be reached.

---

Further addendum: If you actually run the code you have in your question, there is an exception being raised at f.exception() , which as as per the docs :

exception()

Return the exception that was set on this future.

The exception (or None if no exception was set) is returned only if the future is done. If the future has been cancelled, raises CancelledError. If the future isn't done yet, raises InvalidStateError.

This means that the invocation of done_callback() is being stopped at the first if f.exception() . So if you switch done_callback() around to read:

def done_callback(f):
    if f.cancelled():
        _future_set.cancel()
    elif f.exception():
        _future_set.set_exception(f.exception)
    else:
        _future_set.set_result(None)

Then you get the expected output.