Multiple Requests with cURL (PHP)

Question

I am currently trying to scrape this website using cURL in PHP.

http://wateroffice.ec.gc.ca/report/report_e.html?mode=Table&type=realTime&stn=02HD006&dataType=&startDate=2014-09-22&endDate=2014-09-29&prm1=46&prm2=-1

There seems to be a confirmation page that needs to hit before actually viewing the data. On the confirmation page there is a form that gets posted to /includes/disclaimer.php with a post variable of

dislcaimer_action=I Agree

I've tried the following code but the second request seems to always bring me back to the confirmation page.

 $browser = curl_init();
 curl_setopt($browser, CURLOPT_URL, "http://wateroffice.ec.gc.ca/include/disclaimer.php");
 curl_setopt($browser ,CURLOPT_RETURNTRANSFER, true);

 curl_setopt($browser, CURLOPT_HEADER, true);  

 curl_setopt($browser, CURLOPT_FOLLOWLOCATION, 1);

 $postData = 'disclaimer_action=I Agree';

 curl_setopt($browser, CURLOPT_POST, count($postData));
 curl_setopt($browser, CURLOPT_POSTFIELDS, $postData);

 curl_setopt($browser, CURLOPT_URL, "http://wateroffice.ec.gc.ca/report/report_e.html?mode=Table&type=realTime&stn=02HD006&dataType=&startDate=2014-09-22&endDate=2014-09-29&prm1=46&prm2=-1");
 $output = curl_exec($browser);

Not sure what I am doing wrong. Thanks in advance!

Answer 1

You should pass POSTFIELDS as an associative array, just like using $_POST['disclaimer_action'] , but not, in a way.

$postData = array('disclaimer_action' => 'I Agree');
curl_setopt($browser, CURLOPT_POSTFIELDS, $postData);

• The documentation