简体   繁体   中英

NodeJS async callback not succeeding?

I'm trying to implement a scraper using async.map feature to iterate over an array, do some logic and then map them to different values.

async.map($('.listing tr').toArray(), GetLink, function(err, results) {
    if (err) throw err;
    console.log(results);
});

It uses the cheerio module to loop through all table rows to get their links, then completes some logic on them (gets the information from the page it is linked).

var GetLink = function(tablerow, callback) {
    var nameElement = $(tablerow).children().first().find('a');
    var link = nameElement.attr('href');

I then use the 'got' module (a faster/more barebones version of request) to get the pages html.

    got(link, function(error, html, resp) {
        if (!error)
            callback(null, html); // Sends the HTML back as an array result.
    }
}

However the async.map callback [function (err, results)] is never ran even though callback is a valid function from that scope.

Any information would be appreciated.

Such is the way with stack overflow.. you spend a long time trying to fix it yourself, and as soon as you post a question, you figure it out.

The issue was that sometimes there was an error in the got method, which meant that a callback wasn't being run. Therefore I changed:

if (!error)
    callback(null, html);

to

if (!error)
    callback(null, html);
else
    callback(null, null);

Without a guaranteed callback supplied in the logic method, the async.map callback will never be run.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM