使用Apps脚本抓取javascript呈现的网页

Question

I am struggling to put a script together to handle the scraping of a javascript rendered web page through Apps Script. Found this How to scrape Javascript rendered websites using Javascript? here, but I don't know how to put this together. Such as load puppeteer. Any help would be appreciated.

Answer 1

如果您要构建类似于抓取JavaScript生成的内容的内容，建议您遵守使用条款或尝试查找API。

Answer 2

You can try to scrape the initial HTML, since actually scraping the rendered HTML is extremely hard to do, you'd have to use a headless browser.

There is this library: https://github.com/tautologistics/node-htmlparser which you can use to parse HTML from JavaScript, it is in node, but because it doesn't use any dependencies, you can just copy and paste the functions you need.

Parsing it's not a very easy task I'm afraid.