How to delete files older than 7 days but keep last 6 files of 1st of every month in Amazon S3?

Question

We have a backup strategy where every day db dumps are taken and uploaded to s3 bucket. We want to delete backups from s3 which are older than 7 days, but we also want to keep backup of 1st of every month for last 6 months. The backup files are in the format db_dump20220430220001.gz

Answer 1

There is no automatic capability to perform this logic.

However, you could use Tags to accomplish the goal:

• Configure the Amazon S3 bucket to trigger an AWS Lambda function whenever new objects are created in the bucket/path
• Code the Lambda function to add a tag to the object unless it is the first backup of the month (it could do this based purely on the date, or it could look for existing backups in the bucket/path)
• Configure an Amazon S3 Lifecycle rule to delete objects that are older than 7 days AND have the above tag (which means the first backup of each month would not be deleted)
• Add another Lifecycle rule to delete objects older than 6 months (which would delete that backup)