AWS s3同步工作是否需要最小尺寸？

Question

I need to move the contents from the S3 bucket I have been using up until now (src-bucket) to a new S3 bucket within a new AWS account (dst-bucket). As far as I can tell, the straight forward approach would be

aws s3 sync s3://src-bucket s3://dst-bucket --exclude "*" --include="important_prefix*"

But this only works a bit. For a subset of objects I keep getting the error message

An error occurred (AccessDenied) when calling the CopyObject operation: Access Denied

and I cannot find the reason why.

So far I can tell that all the files that cannot be synced are rather small (less than 700kb) while the successfully copied ones are at least a couple of MB each.

Is there some minimum object size (with a relatively high default value) that must be met before s3 sync can do its work?

---

More information

• I can perfectly aws s3 cp and aws s3api get-object the problematic objects. They just fail with aws s3 sync .
• In both accounts I have a user with S3FullAccess permissions

• I added the following bucket policy to src-bucket

   { "Version": "2012-10-17", "Statement": [ { "Sid": "Example permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::newaccountid:user/myadminuser" }, "Action": [ "s3:GetBucketLocation", "s3:ListBucket" ], "Resource": "arn:aws:s3:::src-bucket" }, { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::newaccountid:user/myadminuser" }, "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::src-bucket/*" } ] } 

  • I run aws s3 sync as the newaccountid:user/myadminuser

I can browse around src-bucket with the user from the new account and I can upload and download files just fine. Running the initially mentioned aws s3 sync command copies a few 100 objects from src-bucket to dst-bucket , but fails for some other hundrets of objects.

I tried comparing the not-sync-able objects to those that I was able to sync successfully. I did a aws s3api get-object-acl on many objects in src-bucket . Their ACLs all look the same, for successfully copied ones and copy-failed ones:

{
    "Owner": {
        "ID": "reallylongidihadbashcomparethemformetomakesuretheyareidentical"
    },
    "Grants": [
        {
            "Grantee": {
                "Type": "CanonicalUser",
                "ID": "reallylongidihadbashcomparethemformetomakesuretheyareidentical"
            },
            "Permission": "FULL_CONTROL"
        }
    ]
}

On dst-bucket the really long id is a different one, but apart from that the ACLs look the same.

Answer 1

No, there is no such thing. I was correlating the wrong things. Apparently s3 sync "depends" on being able to copy the objects' tags, as well as their data. Reading objects' tags is a separate privilege from reading the objects data. I was missing the s3:GetObjectTagging action when granting rights to my new user.

Apparently "they" started splitting data into much smaller chunks at the same time they started using object tags.