Archiving AWS RDS mysql Database

Question

I am looking for options to archive my old data from specific tables of an AWS RDS MySQL database. I came across AWS S3, AWS Glacier and copy the data to either one using some Pipelines or Buckets, but from what I understood they copy the data to vault or backups the data, but don't move them.

Is there a proper option to archive the data by moving from RDS to S3 or Glacier or Deep Archive? ie, deleting from the table in AWS RDS after creating an archive. What would be the best option for the archival process with my requirements and would it affect the replicas that already exist?

Answer 1

The biggest consideration when "archiving" the data is ensuring that it is in a useful format should you every want it back again.

Amazon RDS recently added that ability to export RDS snapshot data to Amazon S3 .

Thus, the flow could be:

• Create a snapshot of the Amazon RDS database
• Export the snapshot to Amazon S3 as a Parquet file (you can choose to export specific sets of databases, schemas, or tables)
• Set the Storage Class on the exported file as desired (eg Glacier Deep Archive)
• Delete the data from the source database (make sure you keep a Snapshot or test the Export before deleting the data!)

When you later wish to access the data:

• Restore the data if necessary (based upon Storage Class)
• Use Amazon Athena to query the data directly from Amazon S3

Answer 2

Recently I did build a similar pipeline using AWS lambda that runs on a cron schedule(Cloudwatch event) every month to take a manual snapshot of the RDS, export it to S3, and delete the records that are older than n days

I added a gist of the util class that I used, adding it here if it helps anyone JS Util class to create and export Db snapshots to S3

PS: I just wanted to add it as a comment to the approved answer but don't have enough reputations for that.