cURL not sending POST data

Question

Maybe I just need a pair of fresh eyes....

I need to POST to a page behind .htaccess Basic Authentication. I successfully log in and get past the .htBA, then POST to the target page. I know that the script is getting to that page as I'm logging the access. However $_POST is empty -- evident from both checking the var as well as the target script not working the way it should. (I control all pages).

I've tried many combos of the various curl opts below to no avail. I'm not getting any errors from the second hit.

Thanks.

$post_array = array(
  'username'=>$u,
  'password'=>$p
);


// Login here
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://example.com/admin/login.php');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:43.0) Gecko/20100101 Firefox/43.0');
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath('temp/cookies.txt') );
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath('temp/cookies.txt'));
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_REFERER, 'http://example.com/index.php');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array));
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
  'method' => 'POST',
  "Authorization: Basic ".base64_encode("$username:$password"),
));
$logInFirst = curl_exec ($ch);


/* Don't close handle as need the auth for next page
 * load up a new page */
$post_array_2 = array(
    'localfile'=>'my_data.csv',
    'theater_mode'=>'normal'
    );
//curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath('temp/cookies.txt') );
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath('temp/cookies.txt'));
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_REFERER, 'http://example.com/admin/post_here.php');
curl_setopt($ch, CURLOPT_URL, 'http://example.com/admin/post_here.php');
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array_2));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
  'Content-Type: multipart/form-data;',
  "Authorization: Basic ".base64_encode("$username:$password"),
));
$runAi = curl_exec($ch);
$run_error = curl_error($ch); echo '<hr>'.$run_error.'<hr>';
curl_close($ch);

Here's the code on the target page (post_here.php), which results in a zero count. So I know that the target script is being hit, and based on the output, there are no POSTs.

$pa = '   There are this many keys in POST: '.count($_POST);
foreach ($_POST as $key => $value) {
  $pa .= '   '.$key.':'.$value.'    ----  ';
}

Answer 1

The error is on the second request:

curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array_2));
// ...
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
  'Content-Type: multipart/form-data;',
  // ...

You send the header Content-Type: multipart/form-data but the data is encoded as application/x-www-form-urlencoded (by http_build_query() ).

The data you want to post on the second request contains 'localfile'=>'my_data.csv' . If you want to upload a file on the second request then the content type is correct (but you don't need to set it manually). Don't use http_build_query() but pass an array to CURLOPT_POSTFIELDS , as is explained in the documentation .

Also, for file uploads you have to put a @ in front of the file name and make sure curl is able to find the file. The best way to do this is to use the complete file path:

$post_array_2 = array(
    'localfile'    => '@'.__DIR__'/my_data.csv',
    'theater_mode' => 'normal'
);

The code above assumes my_data.csv is located in the same directory as the PHP script (which is not recommended). You should use dirname() to navigate from the script's directory to the directory where the CSV file is stored, to compose the correct path.

As the documentation also states, since PHP 5.5 the @ prefix is deprecated and you should use the CURLFile class for file uploads:

$post_array_2 = array(
    'localfile'    => new CURLFile(__DIR__'/my_data.csv'),
    'theater_mode' => 'normal'
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array_2);

---

As a side note, when you call curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY); it means curl is allowed to negotiate the authentication method with the server. But you also send the header "Authorization: Basic ".base64_encode("$username:$password") and this removes any negotiation because it forces Authorization: Basic .

Also, in order to negociate, curl needs to know the (user, password) combination. You should always use curl_setopt(CURLOPT_USERPWD, "$username:$password") to tell it the user and password. Manual crafting the Authorization header is not recommended.

If you are sure Authorization: Basic is the method you need then you can use curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC) .

Answer 2

You do not see anything inside post because you are using 'Content-Type: multipart/form-data;', . Just remove that and you should be fine.

If you want to upload a file (ie my_data.csv ) that case you need to follow this way:

## change your file name as following in your param
'localfile'=> '@'.'./my_data.csv',

## after that remove http_build_query() from post
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array_2);

This will automatically add the header multipart with your post.

You may look your uploaded file using $_FILES variable.

Finally, You can observe what curl is enabling verbose mode.

curl_setopt($ch, CURLOPT_VERBOSE, true);

Tips: While using cookie, always close curl after each and every curl_exec() you do. Otherwise it will not probably write things into cookie file after every requests you make!