S3 Bucket Policy and IAM Role Conflicting

Question

I am trying to provide general access to a bucket using an S3 bucket policy, while also allowing specific access to a role using a role policy. The role is used by a Lambda function to handle objects in the bucket. It is being stopped at the first hurdle - it cannot GET anything with the prefix "incoming/", even though it is allowed in the role policy, and not explicitly denied in the bucket policy.

Role Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowBucketPut",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::bucket-name/*"
        },
        {
            "Sid": "AllowIncomingGetDelete",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::bucket-name",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "incoming/*"
                }
            }
        }
    ]
}

Note: I also tried removing the condition and changing the resource to "arn:aws:s3:::bucket-name/incoming*", which only seemed to change how the policy simulator behaved. Another note: GET from the bucket with "incoming/*" prefix does work in the simulator, just not in practice.

I have not removed any statements in the below bucket policy, as I am not sure what might be relevant. IP addresses have been omitted.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicList",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::bucket-name",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "public*"
                }
            }
        },
        {
            "Sid": "AllowPublicGet",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-name/public*"
        },
        {
            "Sid": "AllowPrivateList",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::bucket-name",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "private*"
                },
                "IpAddress": {
                    "aws:SourceIp": [
                        "..."
                    ]
                }
            }
        },
        {
            "Sid": "AllowPrivateGet",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-name/private*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "..."
                    ]
                }
            }
        },
        {
            "Sid": "AllowIncomingPut",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::bucket-name/incoming*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "..."
                    ]
                }
            }
        }
    ]
}

Apologies for the wall of text.

I don't understand why my role is not able to GET objects with the prefix "incoming/".

The Lambda function is getting 403 access denied when doing the following:

S3.download_file(bucket, key, localfile)

Answer 1

According to the documentation ( http://docs.aws.amazon.com/AmazonS3/latest/dev/amazon-s3-policy-keys.html ), the s3:prefix conditions applies to the s3:ListBucket API only, to force the caller to specify a prefix on the ListBucket operation. It doesn't seem to apply to a GetObject API call.

Because of that, your Allow on GetObject with the condition s3:prefix == ... won't match any GET (Object) requests (since those requests don't contain the policy key "s3:prefix"!), so you are effectively not allowing those requests in the Role policy. Since you don't seem to allow that request on the bucket policy either, and there are no Deny statements anywhere, your Lambda code is being implicitly denied .

You should use a Resource instead, as you mentioned you have tried on the policy simulator: "Resource": "arn:aws:s3:::bucket-name/incoming/*" .

Also -- you might have a reason to specify the policies exactly like you did, but it seems a bit unusual --, typically the "Resource" element on S3-related policies, when you want to describe a prefix, will be something like ...incoming/* , instead of just ...incoming* . This could prevent some unexpected results. For instance, say you have a "folder" called incoming/ and later you create a folder called incoming-top-secret/ . The way you wrote the policy, you'd be granting access to both those prefixes! But again - without really knowing the exact details of your environment, it's hard to tell what you truly need. Just wanted to make sure you (and anybody else reading this) is aware of this subtle (but important) detail!

This is all I can think about, based on the description you gave. If you try those changes, and it still doesn't work, please update your question accordingly with the new policies you have tried. Good luck!

Answer 2

Can you try adding the below statement to the bucket policy?

    {
        "Sid": "AllowIncomingGet",
        "Effect": "Allow",
        "Principal": {
            "AWS": "*"
        },
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::bucket-name/incoming/*",
        "Condition": {
            "IpAddress": {
                "aws:SourceIp": [
                    "..."
                ]
            }
        }
    }