What's the proper way of chaining async functions in Node.js?

Question

I have an interesting case where I need to do a few queries in MongoDB using Mongoose, but the response is returning before I can complete all of them.

I have two document types, list and item. In one particular call, I need to get all of the lists for a particular user, then iterate over each of them and fetch all of the items and append them to the appropriate list before returning.

List.find({'user_id': req.params.user_id}, function(err, docs){
  if (!err) {
    if (docs) {

      var results = [];

      _und.each(docs, function(value, key) {

        var list = value.toObject();
        list.items = [];

        Item.find({'list_id': value._id}, function(err, docs) {
          if (!err) {
            _und.each(docs, function(value, key) { list.items.push(value.toObject()); });
            results.push(list);
          }
          else {
            console.log(err);
          }
        });
      });

      res.send(results);

(_und is how I've imported underscore.js)

Obviously the issue are the callbacks, and since there's multiple loops I can't return within a callback.

Perhaps this is a case where I would need to get the count in advance and check it on every iteration to decide when to return the results. This doesn't seem elegant though.

Answer 1

Code solution

First of all the issue is with the code. Your sending the results before the Item.find queries finish. You can fix this quite easily

var count = docs.length + 1;
next()
_und.each(docs, function(value, key) {

    var list = value.toObject();
    list.items = [];

    Item.find({
        'list_id': value._id
    }, function(err, docs) {
        if (!err) {
            _und.each(docs, function(value, key) {
                list.items.push(value.toObject());
            });
            // push asynchronous
            results.push(list);
            next()
        }
        else {
            console.log(err);
        }
    });
});

function next() {
    --count === 0 && finish()
}

function finish() {
    res.send(results)
}​

The easiest way is reference counting, you default count to the number of documents. Then every time your finished getting an item you call next and decrement the count by one.

Once your finished getting all items your count should be zero. Note that we do .length + 1 and call next immediately. This gaurds against the the case where there are no documents, which would otherwise do nothing.

Database solution

The best solution is to use mongo correctly. You should not be doing what is effectively a join in your code, it's slow and inefficient as hell. You should have a nested document and denormalize your list.

so list.items = [Item, Item, ...]

As a further aside, avoid mongoose, it's inefficient, use the native mongo driver.

Answer 2

我使用这个模块： https ： //github.com/caolan/async