Create an array of fetch promises using a for loop with JavaScript/ES6 that can be read via Promise.all?

Question

So, without boring anyone with the backstory, I need to access data from a number of APIs in order to run my script. The data needs to all be loaded before I execute the script, which I'm normally comfortable doing: I just declare some fetch requests, write a Promise.all, then continue on with the function.

HOWEVER, I've hit something of a snafu with a certain API that limits the number of results I can pull from one request to 100 and I need to query all of the results. I didn't think this was a huge deal since I figured I can just make a couple extra requests by affixing "&page=X" to the end of the request.

The plan, then, is to request the total number of pages from the API and then feed that into a for loop to push a number of fetch requests into an array of promises (ie, link://to/api/data&page=1, link://to/api/data&page=2, etc). When I actually attempt to create this array with a for loop, though, the array returns empty. Here's my work:

const dataUrlNoPage = 'link/to/api/data&page=';
const totalPages = 3; //usually pulled via a function, but just using a static # for now

let apiRequestLoop = function(inp) {
  return new Promise(function(resolve){
    let promiseArray = [];
    for (let i = 1; i <= inp; i++) {
      let dataUrlLoop = dataUrlNoPage + i;
      fetch(dataUrlLoop).then(function(response){
        promiseArray.push(response.json());
      })
     }
    resolve(promiseArray);
  })
}

let finalPromiseArray = apiRequestLoop(totalPages).then(result => {
  let requestArray = [apiRequest1,apiRequest2];
  //requestArray contains earlier fetch promises
  result.forEach(foo => {
    requestArray.push(foo);
    }
  );
  return requestArray;
});

So, what's tripping me up is really the loop, and how it's not returning an array of promises. When I look at it in the console, it shows up as a blank array, but I can expand it and see the promises I was expecting. I see the "Value below was evaluated just now" response. No matter how many promises or .thens, I write, however, the array is never actually populated at run time.

What's going on? Can I not generate fetch promises via a for loop?

(Also, just to cut this line of questioning off a bit, yes, the API I'm trying to access is Wordpress. Looking around, most people suggest creating a custom endpoint, but let's assume for the purpose of this project I am forbidden from doing that.)

Answer 1

You have several problems here.

The first is that you have the function provided to new Promise itself containing promise creations. Don't do this! It's a definite anti-pattern and doesn't keep your code clean.

The second is this basic bit of code:

let promiseArray = [];
for (let i = 1; i <= inp; i++) {
  let dataUrlLoop = dataUrlNoPage + i;
  fetch(dataUrlLoop).then(function(response){
    promiseArray.push(response.json());
  })
 }
resolve(promiseArray);

This says:

1. create an empty array
2. loop through another array, doing HTTP requests
3. resolve your promise with the empty array
4. when the HTTP requests are completed, add them to the array

Step four will always come after step three.

So, you need to add the promises to your array as you go along, and have the overall promise resolve when they are all complete.

let apiRequestLoop = function(inp) {
  let promiseArray = [];
  for (let i = 1; i <= inp; i++) {
    let dataUrlLoop = dataUrlNoPage + i;
    promiseArray.push(fetch(dataUrlLoop).then(function(response) {
      return response.json();
    }));
  }
  return Promise.all(promiseArray);
}

or, with an arrow function to clean things up:

let apiRequestLoop = function(inp) {
  let promiseArray = [];
  for (let i = 1; i <= inp; i++) {
    let dataUrlLoop = dataUrlNoPage + i;
    promiseArray.push(fetch(dataUrlLoop).then(response => response.json()));
  }
  return Promise.all(promiseArray);
}

Answer 2

A few points:

• you want to actually put the promises themselves into the array, not push to the array in a .then() chained to the promise.
• you probably want to skip creating a new Promise in your function. Just get an array of all the promises from your loop, then return a Promise.all on the array.

Like this:

let apiRequestLoop = function(inp) {
  let promiseArray = [];
  for (let i = 1; i <= inp; i++) {
    let dataUrlLoop = dataUrlNoPage + i;
     promiseArray.push(fetch(dataUrlLoop))
  }
  return Promise.all(promiseArray);
}

In your final .then statement, in finalPromiseArray, your result will be an array of the results from all the promises. like this [response1, response2, response3, ...]

See the Promise.all documentation for more details.