How to wait for all async calls to finish

Question

I'm using Mongoose with Node.js and have the following code that will call the callback after all the save() calls has finished. However, I feel that this is a very dirty way of doing it and would like to see the proper way to get this done.

function setup(callback) {

  // Clear the DB and load fixtures
  Account.remove({}, addFixtureData);

  function addFixtureData() {
    // Load the fixtures
    fs.readFile('./fixtures/account.json', 'utf8', function(err, data) {
      if (err) { throw err; }
      var jsonData = JSON.parse(data);
        var count = 0;
        jsonData.forEach(function(json) {
          count++;
          var account = new Account(json);
          account.save(function(err) {
            if (err) { throw err; }
            if (--count == 0 && callback) callback();
          });
        });
    });
  }
}

Answer 1

You can clean up the code a bit by using a library like async or Step .

Also, I've written a small module that handles loading fixtures for you, so you just do:

var fixtures = require('./mongoose-fixtures');

fixtures.load('./fixtures/account.json', function(err) {
  //Fixtures loaded, you're ready to go
};

Github: https://github.com/powmedia/mongoose-fixtures

It will also load a directory of fixture files, or objects.

Answer 2

I've recently created simpler abstraction called wait.for to call async functions in sync mode (based on Fibers). It's at an early stage but works. It is at:

https://github.com/luciotato/waitfor

Using wait.for , you can call any standard nodejs async function, as if it were a sync function, without blocking node's event loop. You can code sequentially when you need it.

using wait.for your code will be:

//in a fiber
function setup(callback) {

  // Clear the DB and load fixtures
  wait.for(Account.remove,{});

  // Load the fixtures
  var data = wait.for(fs.readFile,'./fixtures/account.json', 'utf8');

  var jsonData = JSON.parse(data);
  jsonData.forEach(function(json) {
    var account = new Account(json);
    wait.forMethod(account,'save');
  }
  callback();
}

Answer 3

I did a talk about common asyncronous patterns (serial and parallel) and ways to solve them:

https://github.com/masylum/i-love-async

I hope its useful.

Answer 4

That's actually the proper way of doing it, more or less. What you're doing there is a parallel loop. You can abstract it into it's own "async parallel foreach" function if you want (and many do), but that's really the only way of doing a parallel loop.

Depending on what you intended, one thing that could be done differently is the error handling. Because you're throwing, if there's a single error, that callback will never get executed ( count won't be decremented). So it might be better to do:

account.save(function(err) {
  if (err) return callback(err);
  if (!--count) callback();
});

And handle the error in the callback. It's better node-convention-wise.

I would also change another thing to save you the trouble of incrementing count on every iteration:

var jsonData = JSON.parse(data)
  , count = jsonData.length;

jsonData.forEach(function(json) {
  var account = new Account(json);
  account.save(function(err) {
    if (err) return callback(err);
    if (!--count) callback();
  });
});

Answer 5

If you are already using underscore.js anywhere in your project, you can leverage the after method. You need to know how many async calls will be out there in advance, but aside from that it's a pretty elegant solution.