Efficient way to find and delete s3 objects with extension

Question

I have a bucket in S3 for which i want to delete all objects with a particular extension. The easiest solution is to list all keys and checks if it ends with extension and delete it, but this solution is very costly. Can anyone suggest any efficient to achieve this?

Answer 1

Look at S3 Inventory report , if you do not need up-to-the minute accuracy.

Alternatively, you might have to create an index of your S3 objects in DynamoDB or elsewhere so that you can easily find objects with a given suffix. Or even consider restructuring your keys so that they begin with the file extension, then you can list a prefix such as csv/ (obviously this might have negative consequences elsewhere in your application so is not necessarily a good solution).

Note that the price of listing objects in S3 Standard is $0.005 per 1,000 requests and each of those requests will return up to 1,000 S3 keys. I'm not sure how many keys you would be listing but that's $0.005 per million objects.