如何使用 axios 加载 JavaScript 启用的响应或在 Z686155AF75A160A0F6EZE9D8 中获取 API？

Question

I am working on a personal project in which I want to read the whole HTML of a JavaScript dependent webpage. For Example if I to load this URL in a JavaScript Enabled web browser, this is what I get:

However, if I disable JavaScript in the browser, and load the same URL now, I get this:

This is pretty normal I know.

---

Now I am trying to load the HTML of the same link in JavaScript code using axios HTTP client, and obviously I am getting the HTML of JavaScript disabled webpage as the HTTP response.

I want to get the HTML(+JS) source as the response of the same link (in which JavaScript is enabled). I don't know how to mimic a JavaScript enabled Web Browser when working with HTTP clients like axios or fetch API.

Answer 1

If you're trying to do this in the browser, you basically can't unless the site you're loading lets you do so (via CORS or similar). You'd have to load it into a window or iframe , wait for its JavaScript to run, and then access the resulting DOM. But accessing the DOM of a cross-origin page is disabled by default.

The only browser-based way I can think of is to write and install a browser extension , since when a user installs an extension, they can grant greater power to the extension than a web page normally has.

If you're trying to do this in a non-browser environment, you can use a headless browser like headless Chromium or similar. The browser-based restrictions don't apply.