The file upload by CloudFront Origin Access Identity signed url can't be access by boto3 or IAM role?

Question

I followed by cloudfront docuement http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html#private-content-granting-permissions-to-oai for private file.

The bucket policy looks like:

{  
"Version": "2008-10-17",  
"Id": "PolicyForCloudFrontPrivateContent",  
"Statement": [  
    {  
        "Sid": "1",  
        "Effect": "Allow",  
        "Principal": {  
            "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity XXXXXXXXX"  
        },  
        "Action": "s3:*",  
        "Resource": "arn:aws:s3:::XXXXXX/*"  
    }  
]  
}

When I upload file by the signed url with KEY PAIR. The file owner is

Owner CloudFront Origin Access Identity *********

At now, I can't using boto3 in ec2. The command

aws s3 cp s3::/xxx/uploadfile test.txt 

Give me a error:

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

I can upload file which don't use the signed url. These file can be access by boto3 fine. These file owner is

 ****MyCountName***** 

So I can't figure out why ec2 machine can't head the origin access identity file?

Answer 1

As you noticed, when the CloudFront Origin Access Identity (OAI) authorizes the upload, the OAI is the entity that owns the object -- not your account.

Owner CloudFront Origin Access Identity XXXX

OAIs represent an entity that you exclusively control, but they aren't actually part of your AWS account.

The ownership of an object is determined by the account that authorizes the upload, not the account that owns the bucket. Accounts other than the uploading account must be given permission by the account that owns the object.

x-amz-acl: bucket-owner-full-control

http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html

You can make this header mandatory using bucket policy.

If you control the client that makes the uploads, you should be able to add this header.

If you don't control the client, you should be able to add it with a Lambda@Edge Viewer Request trigger. I have not tested this code, but it should accomplish the purpose:

'use strict';

exports.handler = (event, context, callback) => {
  const request = event.Records[0].cf.request;
  if(request.method == 'PUT')
  {
    request.headers['x-amz-acl'] = [
      { key: 'x-amz-acl', value: 'bucket-owner-full-control' }
    ];
  }
  return callback(null, request);
};

Answer 2

Are you trying to upload a file to S3 or download a file from S3? Because you mentioned this command aws s3 cp s3::/xxx/uploadfile test.txt which downloads a file not upload and this is an aws-cli command not boto3.

Please look at this documentation here - http://docs.aws.amazon.com/cli/latest/reference/s3/cp.html