CURL Login and Retrieve Page

Question

I am trying to write a script that will allow me to login to a password protected area of a site running on zencart and grab html in string form. So far it downloads the HTML page, as a .html file, but only the public version of the page (ie it is not logging in successfully). It is HTTPS and I believe this may be part of the reason for why it is not working. What is wrong with my script what CURL settings do I need (there is very little documentation on CURL for PHP sadly :( )

<?php
$username = "xxxx@gmail.com";
$password = "xxxxx";
$securityToken = "b6afe5babdd1b6be234d1976586fb1f1";
$loginUrl =  "https://www.xxxxxxx.com.au/index.php?main_page=login&action=process";

//init curl
$ch = curl_init();

//Set the URL to work with
curl_setopt($ch, CURLOPT_URL, $loginUrl);

// ENABLE HTTP POST
curl_setopt($ch, CURLOPT_POST, 1);

//Set the post parameters
curl_setopt($ch, CURLOPT_POSTFIELDS, 'email_address='.$username.'&password='.$password.'&securityToken='.$securityToken);

//Handle cookies for the login
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie122.txt');

//Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
//not to print out the results of its query.
//Instead, it will return the results as a string return value
//from curl_exec() instead of the usual true/false.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

//execute the request (the login)
$store = curl_exec($ch);

//the login is now done and you can continue to get the
//protected content.

//set the URL to the protected file
curl_setopt($ch, CURLOPT_URL, 'http://www.xxxxxxx.com.au/index.php?main_page=product_info&products_id=1488');

//execute the request
$content = curl_exec($ch);

//save the data to disk
file_put_contents('test122.html', $content);

if(curl_error($ch)) {
$error = curl_error($ch);
echo $error;
}


?>

Answer 1

Try separate url and data. Use http_build_query().

$loginUrl =  "https://www.xxxxxxx.com.au/index.php";
...
$args = array(
  'main_page'     => 'login',
  'action'        => 'process',
  'email_address' => $username,
  'password'      => $password,
  'securityToken' => $securityToken
);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($args));

And you probably need:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

And might be useful:

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

Answer 2

full example working:

 $ch = curl_init($url);

    $headers = array();

    //post
    $post = 'a=b&d=c';
    $headers[] = 'Content-type: application/x-www-form-urlencoded;charset=utf-8';
    $headers[] = 'Content-Length: ' . strlen($post);

    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $post); 

    //if get
   // $headers[] = 'Content-type: charset=utf-8';   

   $headers[] = 'Connection: Keep-Alive';

    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);// Follow redirects
    curl_setopt($ch, CURLOPT_AUTOREFERER, 1);// Set referer on redirect
    curl_setopt($ch, CURLOPT_MAXREDIRS, 5);// Stop after x redirects
    curl_setopt($ch, CURLOPT_TIMEOUT, 20);// Stop after x seconds

    //username and password (if any):
    curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
    curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);

    //https
    $https = strpos($url, "https://");
    if($https !== false){
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    } 

    $response = curl_exec($ch);
    curl_close($ch);

    echo($response);

Answer 3

you have to follow some step to login with https site

Export the certificate.

Go to the site in the browser (FF in my example). Double click the "lock" icon > Security > View Certificate > Details > Export. Save as X.509.

Upload it to where your script can see it.

Then add

    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . "/path/to/certificateCA.crt");