Limit the rate of requests to an API in a CGI script

Question

The script that I'm writing sometimes makes requests to an API and the API requires that requests are limited to a maximum of 1 per second.

What is the most straight forward way of limiting my requests to the API to 1 every second?

Would it involve storing the current time in a file each time a request is made?

Answer 1

You could use a separate thread for the CGI calls and a queuing mechanism that loops with a call to sleep on each iteration.

From 15.3. time

time.sleep(secs) Suspend execution for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal's catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

Answer 2

One can use a rate-limiting python decorator on the function one wishes to rate-limit, like this one from Greg Burek:

import time

def RateLimited(maxPerSecond):
    minInterval = 1.0 / float(maxPerSecond)
    def decorate(func):
        lastTimeCalled = [0.0]
        def rateLimitedFunction(*args,**kargs):
            elapsed = time.clock() - lastTimeCalled[0]
            leftToWait = minInterval - elapsed
            if leftToWait>0:
                time.sleep(leftToWait)
            ret = func(*args,**kargs)
            lastTimeCalled[0] = time.clock()
            return ret
        return rateLimitedFunction
    return decorate

@RateLimited(2)  # 2 per second at most
def PrintNumber(num):
    print num

if __name__ == "__main__":
    print "This should print 1,2,3... at about 2 per second."
    for i in range(1,100):
        PrintNumber(i)