Node.js: socket hang up when requesting

Question

Here's my code:

    request(link, function (error, response, html) {
        if (!error && response.statusCode == 200) {
            console.log(html);
            let $ = cheerio.load(html);
            let article = $('article');
            console.log(article);
            // Get all hyperlinks
            // $(links).each(function(i, link){
            //     console.log($(link).text());
            // });
        } else {
            console.log(error);
        }
    });

I'm getting this following error:

{ Error: socket hang up
    at createHangUpError (_http_client.js:322:15)
    at TLSSocket.socketOnEnd (_http_client.js:425:23)
    at TLSSocket.emit (events.js:187:15)
    at endReadableNT (_stream_readable.js:1094:12)
    at process._tickCallback (internal/process/next_tick.js:63:19) code: 'ECONNRESET' }

When I specifically request from https://otakumode.com/news/5d6cda0d1b5378cb49345630?utm_source=tom&utm_medium=feed&utm_campaign=news_feed ' WHy does this happen and how can I fix it? Thank you so much!

Answer 1

The server is dropping your connection because there's no User-Agent header. If you add that, even something as simple as User-Agent: "Request" , it seems to start working:

const request = require('request');

const link = 'https://otakumode.com/news/5d6cda0d1b5378cb49345630/New-Pok%C3%A9mon-TV-Anime-to-be-Set-in-All-Regions!';

request({
    uri: link,
    headers: {
        "User-Agent": "Request"
    }
}, function (error, response, html) {
    if (!error && response.statusCode == 200) {
        console.log(html);
    } else {
        console.log(error);
    }
});

I have no idea why the server is doing that. Perhaps it's a coding error when there's no User-Agent header or perhaps they just deem it an incompatible request so they just drop the connection. Since browsers would always include a User-Agent header, this issue on their server obviously only affects programmatic requests.