Running functions concurrently in python asyncio

Question

I'm using python 3.6 and trying to use asyncio to run tasks concurrently. I thought asyncio.gather and ensure future would be the tools to use, but it does not seem to be working as I thought it would. Could someone give me pointers?

Here is my code:

import time
import asyncio

async def f1():
  print('Running func 1')
  time.sleep(4)
  print('Returning from func 1')
  return 1

async def f2():
  print('Running func 2')
  time.sleep(6)
  print('Returning from func 2')
  return 2

async def f3():
  print('Running func 3')
  time.sleep(1)
  print('Returning from func 3')
  return 3

async def foo():
  calls = [
    asyncio.ensure_future(f())
    for f in [f1, f2, f3]
  ]

  res = await asyncio.gather(*calls)

  print(res)

loop = asyncio.get_event_loop()
start = time.time()
loop.run_until_complete(foo())
end = time.time()
print(f'Took {end - start} seconds')
print('done')

I would expect the 3 functions to run independently of each other, but each one seems to be blocked behind the other. This is the ouput I get

Running func 1
Returning from func 1
Running func 2
Returning from func 2
Running func 3
Returning from func 3
[1, 2, 3]
Took 11.009816884994507 seconds
done

I would have expected it to take 6 seconds, with the bottleneck being f2.

Answer 1

First off, welcome to StackOverflow.

When you run code inside event loop, this code MUST use async libraries or be run in executor if you want not to block the entire process.

In this way, event loop can send the task background to be executed in a worker or in the event loop itself in case you use async libraries. Meanwhile, event loop can attend new function o portion of code and repeat the same process.

Once any background task has finished, event loop catch them and return its value.

In your case, if you use async library of sleep you should obtain expected results. For example:

  async def f1():
      print('Running func 1')
      await asyncio.sleep(4)
      print('Returning from func 1')
      return 1

Answer 2

我没有阅读本教程，但我希望找到您的解决方案应该很有趣。