Call to wget from PHP's shell_exec not working

Question

I'm trying to run a PHP script locally that scrapes Google with wget and dumps the HTML into temp.html .

Running this command from the terminal works fine:

wget -O temp.html http://www.google.ca

Running this command from PHP also works fine (so it's not a permissions issue):

shell_exec('touch temp.html');

But running this from PHP does not work (does not create temp.html):

shell_exec('wget -O temp.html http://www.google.ca');

Any suggestions? Wrapping that last command in a var_dump() outputs null.

Thanks!

Answer 1

According to man wget , using wget -O temp.html http://google.com takes all documents, concatenates them and prints everything in temp.html , without producing any stdout so PHP's shell_exec doesn't return anything ( null ).

The content of the scraped webpage should be present in temp.html , but shell_exec("wget ...") does not return anything, as not output is produced.

As you mentioned the webpage you are trying to scrape does not work, maybe they implemented some sort of bot-protection preventing exactly what you are trying.

Edit: You may use - to print everything to stdout instead. So try using shell_exec("wget -O - https://google.com"); should return the content of the requested page to your PHP script.

Answer 2

最简单的解决方案是提供wget二进制文件的完整路径，因为运行脚本的用户似乎与您具有相同的$ PATH。

Answer 3

How about using file_put_contents & file_get_contents instead? This should work without having to worry about wget .

 <?php
  $filename = 'temp.html';
  $address = 'http://www.google.ca';
  file_put_contents($filename,file_get_contents($address));
 ?>