How to rename objects boto3 S3?

Question

I have about 1000 objects in S3 which named after

abcyearmonthday1
abcyearmonthday2
abcyearmonthday3
...

want to rename them to

abc/year/month/day/1
abc/year/month/day/2
abc/year/month/day/3

how could I do it through boto3. Is there easier way of doing this?

Answer 1

As explained in Boto3/S3: Renaming an object using copy_object

you can not rename an object in S3 you have to copy object with a new name and then delete the Old object

s3 = boto3.resource('s3')
s3.Object('my_bucket','my_file_new').copy_from(CopySource='my_bucket/my_file_old')
s3.Object('my_bucket','my_file_old').delete()

Answer 2

There is not direct way to rename S3 object. Below two steps need to perform:

1. Copy the S3 object at same location with new name.
2. Then delete the older object.

Answer 3

I had the same problem (in my case I wanted to rename files generated in S3 using the Redshift UNLOAD command). I solved creating a boto3 session and then copy-deleting file by file.

Like

import boto3

session = boto3.session.Session(aws_access_key_id=my_access_key_id,aws_secret_access_key=my_secret_access_key).resource('s3')

# Save in a list the tuples of filenames (with prefix): [(old_s3_file_path, new_s3_file_path), ..., ()] e.g. of tuple ('prefix/old_filename.csv000', 'prefix/new_filename.csv')
s3_files_to_rename = []
s3_files_to_rename.append((old_file, new_file))

for pair in s3_files_to_rename:
    old_file = pair[0]
    new_file = pair[1]

    s3_session.Object(s3_bucket_name, new_file).copy_from(CopySource=s3_bucket_name+'/'+old_file)
    s3_session.Object(s3_bucket_name, old_file).delete()