使用node.js监视服务器端的DOM更改

Question

I'm wondering if its possible to load a website like the twitter search results page using node.js with jsdom and still be able to detect any changes in the DOM after the initial render.

Example:

Initial page

Modified page

Thanks for your help.

Answer 1

If you want to detect what changed in the DOM tree you will need to understand the minified Twitter's javascript code and properly intercept the events or callbacks. I don't suggest you this approach because js code that works perfectly in browser may break when using jsdom. For instance, when I was trying to do some server side scrapping with jquery I got stuck with this bug that in most browsers is definitely not an issue.

For the time being, what I recommend is that you consume the Search API directly from Twitter which is a more straightforward approach.

Hope it helps!

Answer 2

Certainly the best approach is to leverage the Search API (or whatever relevant API) from Twitter to accomplish this.

But if, for some other project, you want to host a full browser environment (JS + dom + renderer) that remains alive and lets you interact with it, the PhantomJS project might work:

http://code.google.com/p/phantomjs/

It hosts a WebKit instance for automation.