Node.js async, but only handle first positive/defined result

Question

What is the best way to create parallel asynchronous HTTP requests and take the first result that comes back positive? I am familiar with the async library for JavaScript and would happy to use that but am not sure if it has exactly what I want.

Background - I have a Redis store that serves as state for a server. There is an API we can call to get some data that takes much longer than reaching the Redis store.

In most cases the data will already be in the Redis store, but in some cases it won't be there yet and we need to retrieve it from the API.

The simple thing to do would be to query Redis, and if the value is not in Redis then go to the API afterwards. However, we'll needlessly lose 20-50ms if the data is not yet in our Redis cache and we have to go to the API after failing to find the data with Redis. Since this particular API server is not under great load, it won't really hurt to go to the API simultaneously/in parallel, even if we don't absolutely need the returned value.

//pseudocode below

async.minimum([

function apiRequest(cb){
   request(opts,function(err,response,body){
         cb(err,body.result.hit);
    }
},
function redisRequest(cb){
   client.get("some_key", function(err, reply) {
          cb(err,reply.result.hit);
     });
}], 

  function minimumCompleted(err,result){

   // this mimimumCompleted final callback function will be only fired once, 
   // and would be fired by one of the above functions - 
   // whichever one *first* returned a defined value for result.hit


 });

is there a way to get what I am looking for with the async library or perhaps promises, or should I implement something myself?

Answer 1

Use Promise.any([ap, bp]) .

The following is a possible way to do it without promises. It is untested but should meet the requirements.

To meet requirement of returning the first success and not just the first completion, I keep a count of the number of completions expected so that if an error occurs it can be ignored it unless it is the last error.

function asyncMinimum(a, cb) {
    var triggered = false;
    var completions = a.length;
    function callback(err, data) {
      completions--;
      if (err && completions !== 0) return;
      if (triggered) return;
      triggered = true;
      return cb(err, data);
    }
    a.map(function (f) { return f(callback); });
}


asyncMinimum([

function apiRequest(cb){
   request(opts,function(err,response,body){
         cb(err,body.result.hit);
    }
},
function redisRequest(cb){
   client.get("some_key", function(err, reply) {
          cb(err,reply.result.hit);
     });
}], 

  function minimumCompleted(err,result){

   // this mimimumCompleted final callback function will be only fired once, 
   // and would be fired by one of the above functions - 
   // whichever one had a value for body.result.hit that was defined

 });

Answer 2

The async.js library (and even promises) keep track of the number of asynchronous operations pending by using a counter. You can see a simple implementation of the idea in an answer to this related question: Coordinating parallel execution in node.js

We can use the same concept to implement the minimum function you want. Only, instead of waiting for the counter to count all responses before triggering a final callback, we deliberately trigger the final callback on the first response and ignore all other responses:

// IMHO, "first" is a better name than "minimum":

function first (async_functions, callback) {
    var called_back = false;

    var cb = function () {
        if (!called_back) {
            called_back = true; // block all other responses

            callback.apply(null,arguments)
        }
    }

    for (var i=0;i<async_functions.length;i++) {
        async_functions[i](cb);
    }
}

Using it would be as simple as:

first([apiRequest,redisRequest],function(err,result){
    // ...
});

Answer 3

Here's an approach using promises. It takes a little extra custom code because of the non-standard result you're looking for. You aren't just looking for the first one to not return an error, but you're looking for the first one that has a specific type of result so that takes a custom result checker function. And, if none get a result, then we need to communicate that back to the caller by rejecting the promise too. Here's the code:

function firstHit() {
    return new Promise(function(resolve, reject) {
        var missCntr = 0, missQty = 2;

        function checkResult(err, val) {
            if (err || !val) {
                // see if all requests failed
                ++missCntr;
                if (missCntr === missQty) {
                    reject();
                }
            } else {
                resolve(val);
            }
        }
        request(opts,function(err, response, body){
            checkResult(err, body.result.hit);
        }

        client.get("some_key", function(err, reply) {
            checkResult(err, reply.result.hit);
        });
    });
}

firstHit().then(function(hit) {
    // one of them succeeded here
}, function() {
    // neither succeeded here
});

The first promise to call resolve() will trigger the .then() handler. If both fail to get a hit, then it will reject the promise.