PHP get_file_contents from aspnet form after POST

Question

I am trying to pull information from the following website: http://www.borsabcn.es/esp/aspx/Empresas/Empresas.aspx

As you can see, this website lists the companies that are on the Barcelona Stock Exchange. But it only lists the first page by alphabetical order. I am working with PHP to automatically get the info from ALL PAGES. For example, if you click on the link toward the top where it says ERC - HUL , then you go to the page which contains more company names starting with the letters: ERC.

I don't have much experience with APSX, but what I could see after investigation, is that when you click that link, the following JS function fires:

var GoPag = function (inicio) {
    document.forms.aspnetForm['ctl00_Contenido_GoPag'].value = inicio;
    document.forms.aspnetForm.submit();
}

This basically just submits the form with a hidden input field ( id="ctl00_Contenido_GoPag" name="ctl00$Contenido$GoPag" ) with a value containing the page number we want to show. The default is 0, and the value for ERC - HUL is 3.

I am trying to get the page corresponding the page 3, I have the following PHP code, which I created after researching a bit:

$postdata = http_build_query(
    array(
        'ctl00$Contenido$GoPag' => '3'
    )
);

$opts = array('http' =>
    array(
        'method'  => 'POST',
        'header'  => 'Content-type: application/xhtml+xm',
        'content' => $postdata
    )
);

$context  = stream_context_create($opts);

$result = file_get_contents('http://www.borsabcn.es/esp/aspx/Empresas/Empresas.aspx', false, $context);

echo $result;

However, this always returns the result for the default page (0) which lists the companies starting with the letter A. It seems liked the posted data isn't being interpreted correctly.

Am I on the right track? Does anyone know what I can do to get the data I am looking for?

Thanks!

Answer 1

I ended up finding a working method using CURL in PHP. I adapted the code from this answer: Php Curl Posting to .aspx

Here is some working code for posting data to the .aspx page and receiving the correct response:

$p = 1; //page number to get

$postdata = http_build_query(
    array(
        '__EVENTTARGET' => '',
        '__EVENTARGUMENT' => '',
        '__VIEWSTATE' => 'your_event_state_value',
        '__EVENTVALIDATION' => 'your_event_validation_value',
        'ctl00$Contenido$GoPag' => $p
    )
);

$url = 'http://www.borsabcn.es/esp/aspx/Empresas/Empresas.aspx';

// Initialise cURL
$ch = curl_init($url);

// Set options (post request, return body from exec)
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);

// Do the request
$result = curl_exec($ch);

//show/check the result
echo $result;

I removed the values for the __VIEWSTATE and __EVENTVALIDATION variables in the above example, but you could find those by investigating the HTML for that page in your browser.