是否可以使用我的本地网络/浏览器从给定网页下载 html 文件，就像我使用 javascript 或 nodejs 自己下载一样？

Question

I'm a bit new to javascriipt/nodejs and its packages. Is it possible to download a file using my local browser or network? Whenever I look up scraping html files or downloading them, it is always done through a separate package and their server doing a request to a given url. How do I make my own computer download a html file as if I did right click save as on a google chrome webpage without running into any server/security issues and errors with javascript?

Answer 1

Fetching a document over HTTP(S) in Node is definitely possible, although not as simple as some other languages. Here's the basic structure:

const https = require(`https`); // use http if it's an http url;

https.get(URLString, res => {
    const buffers = [];
    res.on(`data`, data => buffers.push(data));
    res.on(`end`, ()=>{
        const data = Buffer.concat(buffers);
        /*
        from here you can do what you want with the data. You can write it to a file
        with fs, you can console.log it using data.toString(), etc.
        */
    });
})

Edit: I think I missed the main question you had, give me a sec to add that.

Edit 2: If you're comfortable with doing the above, the way you access a website the same way as your browser is to open up the developer tools (F12 on Chrome) go to the network tab, find the request that the browser has made, and then using http ( s ) .get(url, options, callback) , set the exact same headers in the options that you see in your browser. Most of the time you won't need all of them, all you'll need is the authentication/session cookie.