Error: InvalidAction on DescribeRegions API call (Amazon AWS/EC2 API)

Question

I'm writing an app to interact with the Amazon EC2 API and since I've never done this before, I decided to start with something easy like DescribeRegions.

I'm doing this in C so there are no easy to use libraries out there for this so I'm having to hack it together with libcurl and libcrypto. Full disclosure, this is the first time I'm interacting with AWS/EC2 API programmatically so this may well be a stupid newbie mistake.

I did read through stackoverflow; this is not the same as the question where the person was trying to send the request from bash and hadn't quoted the string. I'm sending the request through curl_easy_perform()

After reading all the documentation I could find (and for this example, let me replace AAAAAAAAA for my AWS Access Key and BBBBBBB for my secret key.

I construct the parameter part of the signing request as described here which reads:

Action=DescribeRegions&AWSAccessKeyId=AAAAAAAA&SignatureMethod=HmacSHA256&"SignatureVersion=2&Timestamp=2013-09-22T02:12:27Z&Version=2013-08-15

and proceed to escape that and generate a signing request of

GET\n
ec2.amazonaws.com\n
/\n
Action%3DDescribeRegions%26AWSAccessKeyId%AAAAAAAAAAAA%26SignatureMethod%3DHmacSHA256%26SignatureVersion%3D2%26Timestamp%3D2013-09-22T02%3A12%3A27Z&Version=2013-08-15

which I then proceed to construct a signature on (let's call it CCCCCCCC)

and come up with a request that reads:

  https://ec2.amazonaws.com/?Action%3DDescribeRegions%26AWSAccessKeyId%3DAAAAAAAAAAAAA%26SignatureMethod%3DHmacSHA256%26SignatureVersion%3D2%26Timestamp%3D2013-09-22T02%3A12%3A27Z&Version=2013-08-15&Signature=CCCCCCCCCCC

When I send this along, I get the following error.

<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message></Error></Errors><RequestID>585f8932-d27b-42b3-b20e-453d8c7ee1ef</RequestID></Response>

The signing mechanism I'm using is a simple hmac_sha256; I also tried the hmac_sha256 library referenced in the wikipedia article and available for download here .

I've verified that my signing algorithm is correct, now I have to only assume that the string that I'm signing is incorrect.

The documentation (AWS Documentation) is unfortunately less than adequate in this regard.

For example, it reads :

Add the query string components (the name-value pairs, not including the initial question mark (?) as UTF-8 characters which are URL encoded per RFC 3986 (hexadecimal characters must be uppercased) and sorted using lexicographic byte ordering. Lexicographic byte ordering is case sensitive.

What exactly are they asking me to sort here?

Any help would be most appreciated. Would it help if I posted complete source code here?

Answer 1

What exactly are they asking me to sort here?

The keys in a set of key/value pairs have no defined sort order, but since there can only be one correct output of the signing algorithm, there can by definition be only one correct input... and the correct input is a string that is constructed by appending the key/value pairs with the keys sorted.

You sort the keys (names) in the query string when building the string to sign. For example, "AWSAccessKeyId" goes before "SignatureMethod" which goes before "Timestamp," etc. You build the string with the keys sorted.

But I think the other issue you have is this:

proceed to escape that and generate a signing request of

...
Action%3DDescribeRegions%26AWSAccessKeyId%AAAAAAAAAAAA ...

Wait. You only urlencode (escape) the keys and the values, not the separators, when building this string. It should look more like this:

Action=DescribeRegions&AWSAccessKeyId= ...

Notice in the examples, the only escaping you see are like those found in the timestamp, where : becomes %3A but the = an & in the query string are not escaped. You'll need to escape the keys and values before building the string, not after.

Answer 2

Thanks to Michael and another answer that I can't see right now, the mysteries are solved.

The complete resolution is this ...

1. as Michael says, the various key/value pairs must be sorted alphabetically and AWSAccessKeyId goes before Action (which is something I was making an error on).
2. I was URI encoding the = and & in the construction of the signing request.
3. HMAC_Final constructs your signature as requested but does not give you a null terminated string; it is up to you to realize that SHA256 is 32 bytes long and to put the null terminator in the right place.

Correct answer credit goes to Michael!