从 S3 IA 中删除 20 亿个对象的最便宜方法

Question

I have a bucket in S3 (Infrequent access) containing 2 billion objects. It is too big to delete in the console or over the api without taking years.

I can create a lifecycle rule to expire and delete the objects but the calculator predicts this will cost me > $20,000 . Is that correct? Is there a better way to delete a bucket?

I have a file effectively containing a list of all the objects in that bucket if that helps.

Update 2021:

An answer below from @MAP points out that there is now an "Empty" button. I haven't tested yet, but looks like the way to go (I'll accept that answer once tested):

Answer 1

If you have a list of all the objects available then you can certainly use Multi Delete Object action. Apparently this API is free . I would create AWS Step Functions state machine to loop through the file and delete 1000 objects at a time. 1000 appears to be the limit.

It will take around 2M step function transactions to delete all the objects in the bucket. As per the pricing for step function it will cost you around $50 + cost of Lambda invocations around $1 so total cost roughly $51.

Update

Using Lambda or Step Functions is probably not the most cost effective option because both ways you will need to read the file (that contains object keys) from some source such as S3. So I think running the script from local machine or any EC2 linux screen appears to be the best option.

Answer 2

Delete operations are for free. You can create a lifecycle Policy to automate a bulk delete.

I would start with a small number of objects first and check billing report to 100% confirm that the delete will not be charged, then go for the rest.

Answer 3

In 2021, anyone who comes across this question may benefit to know that AWS console now provides an empty button.

Select the bucket and click on "empty" button and all objects versioned or not versioned would be emptied/deleted. Depending on the number of objects it can take minutes to days.

Answer 4

Try the S3Browser

It worked well for me when deleted several tens of millions of objects