PUTing files into S3 using Python requests

Question

I've got this URL that was generated using the generate_url(300, 'PUT', ...) method and I'm wanting to use the requests library to upload a file into it.

This is the code I've been using: requests.put(url, data=content, headers={'Content-Type': content_type}) , I've also tried some variations on this but the error I get is always the same.

I get a 403 - SignatureDoesNotMatch error from S3 every time, what am I doing wrong?

Answer 1

Using boto3 , this is how to generate an upload url and to PUT some data in it:

session = boto3.Session(aws_access_key_id="XXX", aws_secret_access_key="XXX")
s3client = session.client('s3')
url = s3client.generate_presigned_url('put_object', Params={'Bucket': 'mybucket', 'Key': 'mykey'})

requests.put(url, data=content)

Answer 2

S3 requires authentication token if your bucket is not public write. Please check token.

I would suggest you to use boto directly.

bucket.new_key()
key.name = keyname
key.set_contents_from_filename(filename, {"Content-Type": content_type})
# optional if file public to read
bucket.set_acl('public-read', key.name)

Also please check did you add Content-Length header. It's required and take part in auth hash calculation.

Answer 3

I ran into the same issue.

It seems the signature had a signed header "host" so it has to be included in the request you send as well.

s3_client = boto3.client(
    's3',
    region_name=<region>,
    config=Config(signature_version='s3v4')
)
kwargs = {
    "ClientMethod": "put_object",
    "Params": {
        "Bucket": <bucket>,
        "Key": <file key>,
    },
    "ExpiresIn": 3600,
    "HttpMethod": "PUT",
}
s3_client.generate_presigned_url(**kwargs)
with open(<my_file>, "rb") as f:
    data = f.read()
response = requests.put(url, data=data, headers={"host": "<bucket>.s3.amazonaws.com"})