S3ResponseError: 403 Forbidden using Boto

Question

I have a permission problem on an S3 bucket when I try to access certain files using BOTO in Python. Here is the bucket policy :

{
    "Version": "2008-10-17",
    "Id": "Policy1407346649831",
    "Statement": [
        {
            "Sid": "Stmt1407346646598",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::029030651757:user/my_iam_user"
            },
            "Action": "s3:*",
            "Resource": ["arn:aws:s3:::my_bucket/*",
                         "arn:aws:s3:::my_bucket"]
        },
        {
            "Sid": "2",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity EFUS443HMBYF"
            },
            "Action": "s3:GetObject",
            "Resource": ["arn:aws:s3:::my_bucket/*",
                         "arn:aws:s3:::my_bucket"]
        },
        {
            "Sid": "3",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity EFUS443HMBYF"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::my_bucket/*"
        }
    ]
}

I have 3 statements. The first one is to authorize the user my_iam_user to access the bucket my_bucket , the second one is to authorize a Cloudfront distribution to read the bucket and the last one is to authorize a Cloudfront distribution to write on the bucket.

Now I have to files on my bucket profile_pictures/15/file1.jpg and profile_pictures/15/file2.jpg . The first one was created using a signed url and CloudFront, the second one was put on the S3 using Boto. Now I try to access the files using Boto. Here is my code:

import boto
from boto.s3.key import Key

s3 = boto.connect_s3(
    aws_access_key_id="access_key_of_my_iam_user",
    aws_secret_access_key="secret_key_of_my_iam_user"
)
bucket = s3.get_bucket("my_bucket", validate=False)

k1 = Key(bucket)
k1.key = "profile_pictures/15/file1.jpg"
k1.get_contents_as_string()


k2 = Key(bucket)
k2.key = "profile_pictures/15/file2.jpg"
k2.get_contents_as_string()

The problem is that the access to file1 returns an error:

S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>8C5DE910C7B18F9E</RequestId><HostId>XiKU5Q+B0Wme3GpUNmUoD9KpUN63T3bFu/rAb/wh3rhDMkykoRsdQIFgyIp8zfAwMR1apbqUEFY=</HostId></Error>

whereas the second one is a success. What could be wrong?

Note: The time on the client that is running the code is good.

Answer 1

Even though the IAM user has been granted full access to the bucket by the first policy, they still will not automatically have access to file1 because they are not the owner of that object (Cloudfront is) and they have not been granted explicit access to the object. Also, presumably, the IAM user is also not the owner of the bucket.

If you look at Example 1 on this page you will see an almost identical situation and further explanation of the of how the object context is used to determine whether a request is granted or denied.

Answer 2

You will not have access to the bucket "my-bucket"..

Bucket names are required to be unique across the entire S3 eco-system.

If you try to access a unique bucket name eg "MY_UNIQUE_ORG_NAME-MY_UNIQUE_BUCKET_NAME" you will probably have better luck..