Control flow and error handling with asynchronous node.js functions

Question

I have a function in which a lot of little processes take place independently, but the result of all of them must go into a final callback. It's not a very unreasonable situation; this kind of code where you don't necessarily know ahead of time what's going to run, when, and for how long is rather common, but after hours in front of this I can't figure out a neat way of doing it. I'm not asking for hours of code time, just a simple explanation on how this could be done.

The wrapper function below completes a series of tasks, asynchronous functions, in a way that it is unclear before execution exactly what will run, when, and how many times. I'm trying to find a neat way of calling a callback ( final callback comment) once all of them are done completing, if they didn't fail first, but exit the wrapper function immediately on failure. An example of this type of situation would be something like that below.

// Definition for wrapper function
function wrapper(callback) {
    asyncfunction1();
    for(/*conditions*/) {
        asyncfunction2();
    }
    asyncfunction3(function(results) {
        asyncfunction4(results);
        for(res in results) {
            asyncfunction5(res);
        }
    });
}

// Call to wrapper function
wrapper(function(err) {                // final callback
    if (err) console.log("Failure");
    else console.log("Success");
});

The wrapper holds a series of functions ( asyncfunction#() ) which must all be completed successfully for the callback to be called as callback(false) . If any one fails, the callback is called then and there as callback(true) and the whole wrapper exits, without closing the process.

My stumbling blocks:

• Is there a way to call the callback upon exit from the wrapper function using a built-in event?
• Is there a native way to exit the wrapper function from several functions deep into it without killing the whole process?

Answer 1

I would also suggest using async.js and you'll have some nested calls. Async was made for this type of thing - control flow in NodeJS is tricky. There are other packages for it (promises, other asynchronous packages, etc).

An example using async is below. Note that the outer .series method could be .parallel if your flow can be run in parallel. I like to pass things around in an outer resultsObj that carries over to each async call. It can hold arrays, values, objects, etc . . . just a holder.

I am also using the .series notation where an object of named functions are passed in, rather than an array of functions. The values passed to the callback are stored, which I've used in asyncFunction4.

var resultsObj = {};
async.series( {
  asyncFunction1: function( seriesCb ) {
    //do stuff
    seriesCb();
  },
  asyncFunction2: function( seriesCb ) {
    async.each( yourArray2, function( item, eachCb ) {
      //do stuff
      eachCb();
    }, function( errEach ) {
      resultsObj.someFlag = true;
      seriesCb( errEach );
    } );
  },
  asyncFunction3: function( seriesCb ) {
    callAsyncFunction3( function( results3 ) {
      async.series( {
        asyncFunction4: function( innerSeriesCb ) {
          var results4 = "yes";
          innerSeriesCb( results4 );
        },
        asyncFunction5: function( innerSeriesCb ) {
          async.each( yourArray5, function( item, eachCb ) {
            //do stuff
            eachCb();
          }, function( errEach ) {
            innerSeriesCb( errEach );
          } );
        }
      }, function( innerSeriesErr, innerSeriesResults ) {
        console.log("Result value of async4 was " + innerSeriesResults.asyncFunction4 );
        seriesCb( innerSeriesErr, innerSeriesResults );
      } );
    } );
  }
}, function( seriesErr, seriesResults ) {
  if ( seriesErr ) console.log( "Failure" );
  else console.log( "Success.  results=" + JSON.stringify( resultsObj ) );
} );

EDIT: One more thing. You'll notice how large this control skeleton is already. And that's without code. Break apart your methods and have them accept (and use) the typical NodeJS callback(err) or callback(err, results) style. Then in your async flows, call them in that shorter way. It will just keep this large controlling file a little more manageable.

function doSomething( input1, input2, callback ){
  if( input1 === input2 ) return callback( new Error("Cannot be equal") );
  callback( null, "Success ");
}

async.series( {
  asyncFunction1: function( seriesCb ) {
    doSomething( 1, 2, seriesCb );
  }
}, function( seriesErr, seriesResults ) {
  if ( seriesErr ) console.log( "Failure" );
  else console.log( "Success" );
} );