How to configure authorization mechanism inline with boto3

Question

I am using boto3 in aws lambda to fecth object in S3 located in Frankfurt Region.

v4 is necessary. otherwise following error will return

"errorMessage": "An error occurred (InvalidRequest) when calling 
the GetObject operation: The authorization mechanism you have 
provided is not supported. Please use AWS4-HMAC-SHA256."

Realized ways to configure signature_versionhttp://boto3.readthedocs.org/en/latest/guide/configuration.html

But since I am using AWS lambda, I do not have access to underlying configuration profiles

The code of my AWS lambda function

from __future__ import print_function
import boto3


def lambda_handler (event, context):
    input_file_bucket = event["Records"][0]["s3"]["bucket"]["name"]
    input_file_key = event["Records"][0]["s3"]["object"]["key"]
    input_file_name = input_file_bucket+"/"+input_file_key

    s3=boto3.resource("s3")
    obj = s3.Object(bucket_name=input_file_bucket, key=input_file_key)
    response = obj.get()
    return event #echo first key valuesdf

Is that possible to configure signature_version within this code? use Session for example. Or is there any workaround on this?

Answer 1

Instead of using the default session, try using custom session and Config from boto3.session

import boto3
import boto3.session
session = boto3.session.Session(region_name='eu-central-1')
s3client = session.client('s3', config= boto3.session.Config(signature_version='s3v4'))
s3client.get_object(Bucket='<Bkt-Name>', Key='S3-Object-Key')

Answer 2

I tried the session approach, but I had issues. This method worked better for me, your mileage may vary:

s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))

You will need to import Config from botocore.client in order to make this work. See below for a functional method to test a bucket (list objects). This assumes you are running it from an environment where your authentication is managed, such as Amazon EC2 or Lambda with a IAM Role:

import boto3
from botocore.client import Config
from botocore.exceptions import ClientError

def test_bucket(bucket):
    print 'testing bucket: ' + bucket
    try:
        s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))
        b = s3.Bucket(bucket)
        objects = b.objects.all()

        for obj in objects:
            print obj.key
        print 'bucket test SUCCESS'
    except ClientError as e:
        print 'Client Error'
        print e
        print 'bucket test FAIL'

To test it, simply call the method with a bucket name. Your role will have to grant proper permissions.

Answer 3

Using a resource worked for me.

from botocore.client import Config
import boto3
s3 = boto3.resource("s3", config=Config(signature_version="s3v4"))
return s3.meta.client.generate_presigned_url(
    "get_object", Params={"Bucket": AIRFLOW_BUCKET, "Key": key}, ExpiresIn=expTime
)