Node multiple get requests loop pass into array

Question

I have a get request in which lies a loop of get requests. They pass back data that I want to add to an array, What would be the best way to structure my code with maybe promises to be able to pass the values back from inside the loop and when they have all finished be able to console.log the data?

app.get('/outer', (res, req) => {
    let url = 'https://samplejson.json';
    https.get(url, (response) => {
      response.on('data', (data) => {
        let parsed = JSON.parse(data);
        let collected = [];
        parsed.forEach((myId) => {


                let innerUrl = `https://samplejson/${innerId}.json`;
                https.get(innerUrl, (innerResponse) => {
                    innerResponse.on('data', (innerData) => {
                        let parsedInner = JSON.parse(innerData);
                        //console.log(JSON.stringify(parsedInner,null,2));
                        let collect = parsedInner.title.trim().split(' ');
                        collect.forEach((w) => {
                            //console.log(w); // works
                            collected.push(w);
                        });
                    }, (err) => res.status(400).send(err));
                 } , (err) => res.status(400).send(err));


        }, (err) => res.status(400).send(err));
        console.log('ending'); // never shown ?
        console.log(collected); // never shown ?
      });
  }, (err) => res.status(400).send(err));
}, (err) => {
    res.status(400).send(err);
});

My issue is that the collected array never gets shown, I am guessing because the requests can never send back the data?

So what is the best way to break down my code and do it the right way? Thanks

Answer 1

The code you've posted above initiate multiple async requests in parsed.forEach() , but console logs the array before the requests have finished. Thus there are no data to display. I've posted a suggested fix below

The code has been made more readable by using the request-promise library.

app.get('/outer', (res, req) => {
  return requestPromise({
      url: 'https://samplejson.json',
      method: 'GET',
      json: true
    })
    .map((myId) => {
      return requestPromise({
          url: `https://samplejson/${myId}.json`,
          method: 'GET',
          json: true
        })
        .then((innerData) => {
          return innerData.title.trim().split(' ');
        });
    })
    .reduce((accumulated, current) => {
      return accumulated.concat( current );
    }, [])
    .then((collected) => {
      console.log("Collected into a flattened array:", collected);
    })
    .catch((error) => {
      res.status(400).send(error);
    });
});