Multiple API connections through Nodejs

Question

I want to connect to theMovieDb api, and fetch a large number of movies. The api limit is 40 calls every 10 seconds and 20 connections per IP.

How do I do this in node.js? Right now I have one http request to the api at a time, but anything more than 40 calls results in errors.

Answer 1

40 requests per 10 second, 20 concurrent connections, so in "one thread" we'll have to wait approximately 5 seconds.

There's an awesome library for all async stuff called async .

var loadOne = function (id, callback) {
   // Do your stuff
   // tell it to wait
   setTimeout(callback, 5000);
} 


async.mapLimit(arrayOfIds, 20, loadOne, function(err, results){
    // results is now an array of stats for each file
});