Perl: Getting all external pings from a site

Question

Sort of new here, so not sure if its an OK question to ask.

I'm pinging a website and that website in turn has javascript that pings other websites with key-value pairs in that other url. Something like websiteA pings websiteB, websiteC with GET requests for (say) state1=3&state2=4&state3=8 etc.

I know I can ping a site with say Net::Ping or use LWP::Simple . But how do I get all other sites a given website pings along with its set of parameters?

If I used a firefox add-on like Live HTTP headers or fiddler I can get what I want, but I would like to automate it from the command line.

Answer 1

It sounds like you need to run the javascript on the initially fetched page.

If you control that page or can be sure it won't change then you could parse the javascript and extract the values for the next stage.

However, easiest might be to use something like PhantomJS which is basically a headless web-browser. You drive it via a javascript script - tell it to fetch the initial page and it will process the javascript on it. You just need to find something on the page that indicates that everything has been pinged (or just wait 30 seconds perhaps).