How to get only the sub folder names from a S3 bucket

Question

I'm having a S3 bucket, which contains many sub folders. let's say,

'test-bucket'
-->'photos'
    -->2020
    -->2019
    -->2018
    ...

In the 'test-bucket' there is a sub folder called 'photos' and in the photos folder there may have multiple sub folders.

So i need to loop through photos bucket and get only the photos older than 2 years (ie from 2018). This is my code below using python library boto3.

current_year = datetime.datetime.now().year
client.list_objects_v2(Bucket='test-bucket',Prefix = 'photos/'+str(current_year-3),MaxKeys=1000 )

But this only returns 2018 data only. Suggest an idea to loop through entire years which are older than 2 years. Thanks in advance

Answer 1

You can obtain the folder names by using ListObjects() while passing a Delimiter value. It then returns the list of folders as CommonPrefixes .

Here's some sample code:

import boto3

s3_client = boto3.client('s3')
objects = s3_client.list_objects_v2(Bucket='my-bucket', Delimiter='/', Prefix ='photos/')

for prefix in objects['CommonPrefixes']:
    print(prefix['Prefix'])