How to include the static HTML results of a dynamic Javascript page in PHP?

Question

I have a small script that pulls HTML from another site using Javascript.

I want to include that static HTML that gets pulled in a PHP page without any of the Javascript code appearing in the final PHP page that gets displayed.

I tried doing an include of the file with the Javascript code in the PHP page, but it just included the actual Javascript and not the results of the Javascript.

So how would I go about doing this?

Answer 1

You would need to fetch the page, execute the JavaScript in it, then extract the data you wanted from the generated DOM.

The usual approach to this is to use a web automation tool such as Selenium .

Answer 2

You simply can't.

You need to understand that PHP and Javascript operate on different places, PHP on the server and Javascript on the client.

Your only solution is to change the way all this is done and use "file_get_contents(url)" from PHP to get the same content your javascript used to get. This way, there is no javascript anymore and you can still pre-process your page with distant content.

Answer 3

You wouldn't be able to do this directly from within PHP, since you'd need to run Javascript code.

I'd suggest passing the URL (and any required actions such as click event, etc) to a headless browser such as Phantom or Zombie , and capturing the DOM from it once the JS engine has done it's work.

You could also use a real browser, but of course you don't need a UI in your case, and it might actually get in the way of what you're trying to do, so a headless browser might be better.

This sort of thing would normally be used for automated testing of a site (ie Functional Testing).

There is a PHP tool named Mink which can run these sorts of scripts from within a PHP program. It is aimed at writing test scripts, but I would imagine you could use it for your purposes.

Hope that helps.