Simplify a rate limiting algorithm for Node.js server

Question

I came up with a naive solution for a rate limiting algorithm for a Node.js server and I believe there is a way to simplify it, but I am not sure how yet.

We want to limit requests to 50 per second. So if the newest request comes in and the time between the newest request and request that is 50 places back is less than one second, we should reject the new request.

The naive way to implement this would be to have a simple array that contains 50 timestamps. Every-time an event comes in we assign it a value of Date.now() / process.hrtime(). We then look at the timestamp value of the 50th (last) timestamp in the queue and the new request's Date.now() value, and if the difference in timestamps is greater than 1 second, then we accept the new request and unshift it onto the "queue", and pop the oldest timestamp off the queue. However, if the difference is less than 1 second, we must reject the request, and we don't unshift it onto the queue, and we don't pop the oldest timestamp off.

Here is the code I have on an Express server

var mostRecentRequestsTimestamps = [];

app.use(function(req,res,next){

    if(req.baymaxSource && String(req.baymaxSource).toUpperCase() === 'XRE'){

        var now = process.hrtime(); //nanoseconds

        if(mostRecentRequestsTimestamps.length < 50){
            mostRecentRequestsTimestamps.unshift(now);
            next();
        }
        else{
            var lastItem = mostRecentRequestsTimestamps.length -1;
            if(now - mostRecentRequestsTimestamps[lastItem] < 1000){  // 1000 milliseconds = 1 second
                res.status(503).json({error: 'Server overwhelmed by XRE events'});
            }
            else{
                mostRecentRequestsTimestamps.pop();
                mostRecentRequestsTimestamps.unshift(now);
                next();
            }
        }
    }
    else{
        next();
    }

});

As you could see, it only blocks events if they are coming from one particular source, so it shouldn't starve other types of requests. This logic requires a data structure of 50 timestamps, which is basically nothing, but I would like a way to simplify this even further if possible. Anyone have any ideas? thanks

Answer 1

Here's the simplest I can make it:

// oldest request time is at front of the array
var recentRequestTimes = [];
var maxRequests = 50;
var maxRequestsTime = 1000;

app.use(function(req,res,next){
    if(req.baymaxSource && String(req.baymaxSource).toUpperCase() === 'XRE'){
        var old, now = Date.now();
        recentRequestTimes.push(now);
        if (recentRequestTimes.length >= maxRequests) {
            // get the oldest request time and examine it
            old = recentRequestTimes.shift();
            if (now - old <= maxRequestsTime) {
                // old request was not very long ago, too many coming in during that maxRequestsTime
                res.status(503).json({error: 'Exceeded 50 requests per second for XRE events'});
                return;
            }
        }
    }
    next();
});

This is conceptually different than your implementation in two ways:

1. I use the recentRequestTimes array in increasing order (just made a lot more logical sense to my programming brain)
2. I always add every request to the array, even when it's overloaded. You were not counting requests that hit overload which I think is wrong. This also simplifies the code since you can just add the current time at the start of the function in one place and then just process the array.