Asyncio & rate limiting

Question

I writing an app based on the asyncio framework. This app interacts with an API that has a rate limit(maximum 2 calls per sec). So I moved methods which interact with an API to the celery for using it as rate limiter. But it is looks like as an overhead.

There are any ways to create a new asyncio event loop(or something else) that guarantees execution of a coroutins not more then n per second?

Answer 1

The accepted answer is accurate. Note however that, usually, one would want to get as close to 2QPS as possible. This method doesn't offer any parallelisation, which could be a problem if make_io_call() takes longer than a second to execute. A better solution would be to pass a semaphore to make_io_call, that it can use to know whether it can start executing or not.

Here is such an implementation: RateLimitingSemaphore will only release its context once the rate limit drops below the requirement.

import asyncio
from collections import deque
from datetime import datetime

class RateLimitingSemaphore:
    def __init__(self, qps_limit, loop=None):
        self.loop = loop or asyncio.get_event_loop()
        self.qps_limit = qps_limit

        # The number of calls that are queued up, waiting for their turn.
        self.queued_calls = 0

        # The times of the last N executions, where N=qps_limit - this should allow us to calculate the QPS within the
        # last ~ second. Note that this also allows us to schedule the first N executions immediately.
        self.call_times = deque()

    async def __aenter__(self):
        self.queued_calls += 1
        while True:
            cur_rate = 0
            if len(self.call_times) == self.qps_limit:
                cur_rate = len(self.call_times) / (self.loop.time() - self.call_times[0])
            if cur_rate < self.qps_limit:
                break
            interval = 1. / self.qps_limit
            elapsed_time = self.loop.time() - self.call_times[-1]
            await asyncio.sleep(self.queued_calls * interval - elapsed_time)
        self.queued_calls -= 1

        if len(self.call_times) == self.qps_limit:
            self.call_times.popleft()
        self.call_times.append(self.loop.time())

    async def __aexit__(self, exc_type, exc, tb):
        pass


async def test(qps):
    executions = 0
    async def io_operation(semaphore):
        async with semaphore:
            nonlocal executions
            executions += 1

    semaphore = RateLimitingSemaphore(qps)
    start = datetime.now()
    await asyncio.wait([io_operation(semaphore) for i in range(5*qps)])
    dt = (datetime.now() - start).total_seconds()
    print('Desired QPS:', qps, 'Achieved QPS:', executions / dt)

if __name__ == "__main__":
    asyncio.get_event_loop().run_until_complete(test(100))
    asyncio.get_event_loop().close()

Will print Desired QPS: 100 Achieved QPS: 99.82723898022084

Answer 2

I believe you are able to write a cycle like this:

while True:
    t0 = loop.time()
    await make_io_call()
    dt = loop.time() - t0
    if dt < 0.5:
        await asyncio.sleep(0.5 - dt, loop=loop)