如何为Amazon S3存储桶中的每个文件夹执行命令（如副本）？

Question

I have an S3 bucket with roughly a thousand folders. Inside each folder, I want to copy a file to a new folder a level in. Something like:

foreach (folder in myBucket){
  cp folder/myFile.abc folder/myNewFolder/myFile.abc;
}

How can I execute this one-off logic in S3?

Answer 1

I ended up just using bash and Amazon's own CLI :

BUCKET=myBucketName
for dir in $(aws s3 ls $BUCKET | awk '{print $2}'); do
    aws s3 cp s3://$BUCKET/${dir}myFile.abc s3://$BUCKET/${dir}myNewFolder/myFile.abc
done

aws s3 ls does not include any helpful flags to modify its output, and so I used awk to cut off the "PRE" that came with the directory output.

Place the above in myScript and execute it with bash myScript .