How to copy files from a bucket on one AWS account to a bucket on another AWS account?

Question

I am facing a change in project requirement where until now I had to copy files between 2 buckets under the same AWS account, and now I need to copy the files into a bucket in another account.

Current code:

public void copyEventFromS3(String fromBucket, String fromKey, String toBucket, String toKey) {
    ObjectListing listObjects = this.getListObjects(fromBucket, fromKey);
    listObjects.getObjectSummaries().stream().forEach((item) -> {
        this.getS3().copyObject(item.getBucketName(), item.getKey(), toBucket, toKey);
    });
}

Of course I have the two account credentials.

How can this be done?

Answer 1

In order to copy between buckets using S3's CopyObject API, the (single) role performing the copy must have read access to the source and write access to the destination buckets . The fact that they are in different accounts shouldn't make a difference, once you set up the permissions properly. In addition to the IAM Role's Permission Policy allowing the appropriate read and write access, the buckets must have Bucket Policies allowing the access to that role.

If you can't set up a single role, the other option would be to set up separate S3 Clients (one to read from one account, the other to write to the other) and you would need to manually download and upload each file. This would likely take significantly more time, require large amounts of bandwidth, and may cost more.

Answer 2

If you want to use the copyObject method, you'll need one set of credentials that will give you permissions on both buckets.

So best to use an IAM user from the target account, and give it read permissions on the S3 bucket in the source account.

There's a walkthrough here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-walkthroughs-managing-access-example2.html