使用 Node.js 监控第三方流量的最佳方式是什么？

Question

I'm trying to identify all outbound http requests for resources on third-party sites using Node.js. I've started using Phantomjs (and phantomjs-node) which has network monitoring features, but I'm finding it unreliable for many sites.

I've seen tools like Charles and Fiddler, but I'm trying to incorporate the third-party http requests into my own Node.js app.

Is there any other way to approach this without using a headless browser?

Answer 1

I am not really a fan of using phantomjs from Node. Essentially PhantomJS has its own Event Loop and Node JS has its own and most of the npm modules(including phantomjs-node) available online will use some sort of messaging(Websocket etc.) to communicate between the two.

Now back to your original problem.

You have a few options:

1. If you want to just capture requests from a web page for a few executions you may as well use a Chrome or Firefox extension. In Firefox you can Use Firebug( http://getfirebug.com/ ) to capture net and XHR requests. In Chrome you can use the Chrome Dev Tools to capture requests and right click Save the request as JSON.
2. You can write your own Firefox extension which will capture the Net requests and log them.
3. If you want to use PhantomJS I would recommend using PhantomJS 2( since it uses the latest version of WebKit). I would also recommend using it directly instead of from NodeJS just to reduce the complexities.