Calling asynchronous function in callback

Question

I'm having some trouble understanding asynchronous functions. I've read the chapter in Mixu's Node Book but I still can't wrap my head around it.

Basically I want to request a ressource (using the node package cheerio ), parse it for valid URLs and add every match to my redis set setname .

The problem is that in the end it's only adding the first match to the redis set.

function parse(url, setname) 
{
    request(url, function (error, response, body) 
    {
        if (!error && response.statusCode == 200) 
        {
            $ = cheerio.load(body)

            // For every 'a' tag in the body
            $('a').each(function() 
            {
                // Add blog URL to redis if not already there.
                var blog = $(this).attr('href')
                console.log("test [all]: " + blog);

                // filter valid URLs
                var regex = /http:\/\/[^www]*.example.com\//
                var result = blog.match(regex);
                if(result != null) 
                {
                    console.log("test [filtered]: " + result[0]);

                    redis.sismember(setname, result[0], function(err, reply) 
                    {
                        if(!reply) 
                        {
                            redis.sadd(setname, result[0])
                            console.log("Added " + result[0])
                        }
                        redis.quit()    
                    })
                }
            })
        }
    })
}

I'd be very grateful for pointers on how I'd have to restructure this so the redis.sadd method is working with the correct result.

The output of the current implementation looks like:

test [all]: http://test1.example.com/
test [filtered]: http://test1.example.com/
...
Added http://test2.example.com/

So it's adding the test1.example.com but not printing the "added" line, and it's not adding the test2.example.com but it's printing the "added" line for it.

Thank you!

Answer 1

The first issue is caused by redis.sismember() being asynchronous: when its callback is called, you have already overwritten the result variable so it will point to the last value it had, and not the value at the moment at which you called redis.sismember() .

One way to solve that is to create a new scoped variable by wrapping the asynchronous function in a closure:

(function(result) {
  redis.sismember(setname, result[0], function(err, reply) {
    ...
  });
})(result);

Another option is to create a partial function that's used as callback:

  redis.sismember(setname, result[0], function(result, err, reply) {
    ...
  }.bind(this, result));

The second issue is, I think, caused by redis.quit() being called, which closes the Redis connection after the first sadd() . You're not checking err , but if you do it might tell you more.