cURL redirect request to amazon s3 returns 400 code

Question

Using PHP curl to fetch a release tag file from github. Github responds with a redirect header to S3 which cURL does follow. For example:

https://github.com/twbs/bootstrap/releases/download/v3.3.1/bootstrap-3.3.1-dist.zip

Results in a 302 with a location header to s3 with various authentication details

  https://s3.amazonaws.com/github-cloud/releases/2126244/558cd3fe-6a4b-11e4-9a0e-d0e8b6837eb8.zip?response-content-disposition=attachment%3b%20filename%3dbootstrap-3.3.1-dist.zip&response-content-type=application/octet-stream&awsaccesskeyid=akiaistnzfovbijmk3tq&expires=1417038428&signature=rn98zc%2bxrlhdjpbrulsptthsyee%3d

Following the redirect location header, Amazon responds with 400

Request specific response headers cannot be used for anonymous GET requests.

As an experiment I've tried sending the redirect location as POST and as well as using CURLOPT_CUSTOMREQUEST set to GET and placing the GET parameters in CURLOPT_POSTFIELDS. Those result in 405 The specified method is not allowed against this resource.

The PHP code is basic cURL and sets the following:

CURLOPT_RETURNTRANSFER  => 1,
CURLOPT_HEADER          => 1,
CURLOPT_FOLLOWLOCATION  => 0,
CURLOPT_CONNECTTIMEOUT  => 10,
CURLOPT_TIMEOUT         => 10,
CURLOPT_MAXREDIRS       => 3,
CURLOPT_ENCODING        => 'gzip,deflate',
CURLOPT_POST            => 0,

It also sets

CURLOPT_REFERER

on the redirect

Doing a straight curl -L on a *nix command line works fine though, so it appears to be something with how PHP is handling, or not, the redirect details

Answer 1

The URL you've pasted is not valid, as shown, because three of the query string parameters have been converted -- somehow -- to lower case. They should be...

AWSAccessKeyID
Expires
Signature

The values, too, from the AWSAccessKeyID and Signature have been coerced to lower case, which is also wrong. They are definitely case-sensitive, too.

The response-content-* parameters are correct, as shown.

I'm not a php expert, so I have no idea how this could be happening, but having the case wrong on those case-sensitive parameter names would cause exactly the error you're seeing, since S3 thinks the request is anonymous, and anonymous requests do not support setting response attributes via the query string.

This also strikes me as incorrect, but it's not clear why, in light of this, the redirect would be followed at all:

 CURLOPT_FOLLOWLOCATION => 0,