How to authenticate session in application running on Beanstalk with boto3

Question

My app is deployed via Elastic Beanstalk. It needs to access S3. I can do it locally with my own access key, but I don't want to store that anywhere when I deploy. Given that the instance is on Beanstalk, there must be an easier way to auth , perhaps with roles ?

I have given full S3 permissions to the role used on the Beanstalk instance but I don't know how to set up the Session .

How can I replace this?:

session = boto3.session.Session(
        aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
        aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY'))
client = session.client('s3')
s3 = session.resource('s3')
bucket = s3.Bucket(os.environ.get('S3_BUCKET'))
# do stuff

Answer 1

The recommended way of managing credentials used to sign API requests to other AWS services is using IAM roles . When an IAM role is attached to an instance, it retrieves a temporary credentials from the instance metadata. These credentials are valid for a limited period of time, however SDK manages them transparently. So, instead of creating and distributing your AWS credentials to instance, you can delegate permissions using IAM role.

When creating the IAM role, in addition to access policies, you have to attach a trust policy (eg what service can assume this role) as well.

Assume role policy

An assume role policy (also called as a trust policy) is a policy that grants an access to AWS service to use (assume) that particular role. So, if you are using EC2 instance, a trust policy could look like:

{
    "Action": "sts:AssumeRole",
    "Effect": "Allow",
    "Principal": {
        "Service": "ec2.amazonaws.com"
    }
}

Access policy

The access policy on another hand, grants an access to IAM role to specific AWS resources. So, for example the policy for full access to S3 service would look like

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["s3:*"],
            "Resource": ["*"]
        }
    ]
}

Once you have a role created and attached to particular instance, you can use SDK without supplying any credential or region to it and use it in your code like

s3 = boto3.resource('s3')
bucket = s3.Bucket(os.environ.get('S3_BUCKET'))

Answer 2

The cleanest way is to use IAM roles .

By default, elastic beanstalk instances have two types of IAM roles attached:

• A "service role" : aws-elasticbeanstalk-service-role
• An "IAM Instance Profile" : aws-elasticbeanstalk-ec2-role

Basically the later is what grants your instances rights.

There are several ways modify access rights for instances to use your AWS ressources. If you want Application-specific access (ie only grant right to a given elastic beanstalk application, not to all of them) you can do the following:

• Create (or find) an IAM policy (in IAM console) that has the specific rights you want (eg AmazonS3FullAccess )
• Create an IAM role for EC2 which will replace the default aws-elasticbeanstalk-ec2-role . In the console, you can create a New EC2 role, and attach to it both the policies included in the standard aws-elasticbeanstalk-ec2-role and your additional rights.

• In the elastic beanstalk console of your application, go to configuration > Security > Edit and select your new role in place of the default aws-elasticbeanstalk-ec2-role .

• Apply changes (which will normally respawn your instances).

Your good to go, make sur your script does't has any other authentication method (like config files etc) to make sure boto3 uses the instance IAM profile. then you can call s3 like s3 = boto3.resource('s3')