How to do `PUT` on Amazon S3 using Python Requests

Question

I am trying to upload a file to Amazon S3 with Python Requests (Python is v2.7.9 and requests is v2.7). Following the curl command which works perfectly:

curl --request PUT --upload-file img.png https://mybucket-dev.s3.amazonaws.com/6b89e187-26fa-11e5-a04f-a45e60d45b53?Signature=Ow%3D&Expires=1436595966&AWSAccessKeyId=AQ

But when I do same with requests, it fails. Here's what I have tried:

url = https://mybucket-dev.s3.amazonaws.com/6b89e187-26fa-11e5-a04f-a45e60d45b53?Signature=Ow%3D&Expires=1436595966&AWSAccessKeyId=AQ
requests.put(url, files={'file': base64_encoded_image})
requests.put(url, files={'upload_file': base64_encoded_image})

It fails with 403 and response I am getting is:

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

Then I ran curl in verbose mode:

* Hostname was NOT found in DNS cache
*   Trying 54.231.168.134...
* Connected to mybucket-dev.s3.amazonaws.com (54.231.168.134) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: *.s3.amazonaws.com
* Server certificate: VeriSign Class 3 Secure Server CA - G3
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> PUT /6b89e187-26fa-11e5-a04f-a45e60d45b53?Signature=Ow%3D&Expires=1436595966&AWSAccessKeyId=AQ HTTP/1.1
> User-Agent: curl/7.37.1
> Host: mybucket-dev.s3.amazonaws.com
> Accept: */*
> Content-Length: 52369
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< x-amz-id-2: 5lLCQ3FVrTBg2vkyk44E+MecQJb2OGiloO0+2pKePtxPgZptKECNlUyYN43sl4LBNe9f8idh/cc=
< x-amz-request-id: 636A24D53DEB5215
< Date: Fri, 10 Jul 2015 12:04:44 GMT
< ETag: "5802130d4320b56a72afe720e2c323a7"
< Content-Length: 0
* Server AmazonS3 is not blacklisted
< Server: AmazonS3
<
* Connection #0 to host mybucket-dev.s3.amazonaws.com left intact

So then I added headers

headers = {'Content-Length': '52369', 'Host': 'mybucket-dev.s3.amazonaws.com', 'Expect': '100-continue', 'Accept': '*/*', 'User-Agent': 'curl/7.37.1'}
requests.put(url, files={'file': base64_encoded_image}, headers=headers)

I tried with different combinations of header, still it throws same error. Then I tried to send query parameters too:

payload={'Expires': '1436595966', 'AWSAccessKeyId': 'AQ', 'Signature': 'Ow%3D'}
requests.put(url, files={'file': base64_encoded_image}, headers=headers, data=payload)

It still fails and same error. I tried URL without query parameters and sending them as data=payload to requests, it fails with same error.

Answer 1

Engineers at requests helped me:

with open('img.png', 'rb') as data:
    requests.put(url, data=data)

Answer 2

Acording to this documentation you have pass the files argument to post method also need send the key name for S3

import requests

url = 'https://s3.amazonaws.com/<some-bucket-name>'

data = { 'key': 'test/test.jpeg' }
files = { 'file': open('test.jpeg', 'rb') }

r = requests.post(url, data=data, files=files)
print "status %s" % r.status_code