How to get return value of multiple async calls when it is all done?

Question

I am using 3rd party api method that is async. I have list of items that need to be passed onto this async method and I would like to print out results of all returning results once all async calls are completed. I know I can use callback to accomplish this. But I cannot get it to work. It prints nothing. Obviously I am using callback wrong here. Yes I read about callback here : https://github.com/maxogden/art-of-node#callbacks . Has good examples. But not sure how to make it work with array of async calls and combining the results.

var resultArray = [];
var items = ["one", "two", "three"];
getResult(items, printResult);
function printResult()
{
  for(let j=0; j < resultArray.length; j++)
  {
      console.log(resultArray[j]);
  }
}
function getResult(items, callback)
{
 for(let i=0; i<items.length; i++)
 {
  apiClient.findItem(items[i], function (error, item){
  resultArray.push(item.key);
  });
 }
}

Answer 1

As @JeffreyAGochin pointed out, you can substitute this with promises. If you don't want to do that and you want to stick with callbacks (which I would not recommend), you can use the excellent async .

function getResult(item, done) {
  apiClient.findItem(item, done);
}

async.each(items, getResult, function (error, results) {
  // if error is null, then all of your results are in 'results'
  if(error !== null) throw error;
  results.forEach(function(result) {
    console.log(result);
  });
});

Example promise implementation (I'm assuming you're using ES6 and thus have Promise natively due to your let )

// When you are using promises natively (apparently these have performance implications, see comments) your code looks like this:

function getResult(item) {
  return new Promise(function(resolve, reject) {
    apiClient.findItem(item, function(error, foundItem) {
      if(error) return reject(error);    
      resolve(foundItem);
    });
  });
}

// If you use the excellent bluebird library though (which is pretty common actually), it looks more like this.
let apiClient = Bluebird.promisifyAll(apiClient);
function getResult(item) { return apiClient.getItemAsync(item); }

var resultsPromise = Promise.all(items.map(getResult));
resultsPromise.then(function(results) {
  results.forEach(function(result) {
    console.log(result);
  });
});

As for the reason why so many people are suggesting promises; it's because they compose far, far better. Also there are some great libraries that support promises such as highland (which is also by the same author as async above) which treats a promise as a first class value. It's hard to treat callbacks like this because there's no real way to 'pass them around'

Answer 2

+1 for promises. Another option preferred by some is the async module:

var async = require('async');

async.each(
  items, 
  function (item, cb) { apiClient.findItem(item, cb); },
  function (err, resultArray) {
    for (let j=0; j < resultArray.length; j++) {
      console.log(resultArray[j]);
    }        
  }
);

Answer 3

I would suggest using a Promises library to manage all of the async requests. They typically have a all method, that waits for all promises to complete. You can wrap the other async library call in the promises.

https://github.com/kriskowal/q

Answer 4

In getResult you're not calling the callback parameter, so it will definitely not print anything in that case.

If you're sticking to callbacks instead of promises, which is fully acceptable despite the raging opinion-wars, a nice npm package is Async . Check out the each function in that library, it will probably do what you want.

Answer 5

I think this is what you are looking for: http://howtonode.org/promises .

I suggest you use a variable to count the requests done, so you can check when all the them are finished and do what you need with the array.