使用 AJAX 请求抓取页面

Question

I'm using Node.js with jsDOM to scrape web pages and execute JavaScript in them. But if a page contains AJAX requests, they are not executed.

Is there an option to execute AJAX calls inside the scraped pages?

Answer 1

you need to emulate XMLHTTPRequests. Here's an abstraction . They don't work out of the box in node.js.

You will have to figure out how to get it to talk to jsdom. (Hopefully just appending a xmlhttprequest constructor to the window will fix it)