Node.js - how to call a Facebook API within a for loop without exceeding rate limit?

Question

I have an array with almost 2 millions of Facebook idSender, and I want to itearate over it, calling a Facebook API for each one of them. Now, due to the asynchronicity, if I launch a simple for loop, Facebook would return me a rate limit exceed error after 4-5 minutes. After some attempts I found out that the ideal delay to avoid rate limit is 20 milliseconds.

So I tried this solution: I wrapped my function in a Promise, and I used setTimeout to set a delay.

async function asyncLoop(values) {
  var prm = await new Promise(function(resolve, reject) {
    setTimeout(function() {
        for (var i=0; i<values.length; i++) {
            check(values[i].idSender, values[i].iscrizione, values[i].id_sito)
            if(checked == values.length) {
                resolve()
            }
        }
    },20);
  });
  return prm;
}

asyncLoop(idSenderArray)

but this solution it's not working, and I'm also aware it's almost never a good idea using setTimeout fuction to manipulate asynchronicity in Javascript, in fact I can't even say what exactly it's happening there.

Any suggestion for possible modifies to my function? Any idea on a totally new implementation that could work? Thanks

Answer 1

Just await a time inside the for loop:

const timer = ms => new Promise(res => setTimeout(res, ms));

async function asyncLoop(values) {
  for (var i = 0; i < values.length; i++) {
    await timer(20);
    check(values[i].idSender, values[i].iscrizione, values[i].id_sito)
  }
}

Answer 2

You may also do something like this with promises;

 var apiCall = (url,n) => new Promise((v,x) => setTimeout(v,50,`the result of API call @ ${url} no ${n}`)), calls = Array(20).fill() .map((_,i) => new Promise((v,x) => setTimeout(v,20*i)) .then(_ => apiCall("http://facebook.com",i)) .then(v => console.log(`I am doing something with ${v}`))); 

 .as-console-wrapper{ height: 100%; max-height: 100% !important } 

Answer 3

Alternatively, using setTimeout with promises can also be used:

async function asyncLoop(values) {
    let apiCallPromises = values.map((value, index)=> new Promise(resolve=>{
        setTimeout(_=>{
            check(value.idSender, value.iscrizione, value.id_sito)
            resolve()
        }, index*20)
    }))

    return Promise.all(apiCallPromises);
}

Fairly strait-forward, it maps each value to a check() call with a delay of 20n ms for each subsequent request.