Upload only newly modified files to S3 bucket using Golang aws-sdk

Question

I'm trying to implement a backup mechanism to S3 bucket in my code. Each time a condition is met I need to upload an entire directory contents to an S3 bucket.

I am using this code example:

https://github.com/aws/aws-sdk-go/tree/c20265cfc5e05297cb245e5c7db54eed1468beb8/example/service/s3/sync

Which creates an iterator of the directory content's and then use s3manager.Upload.UploadWithIterator to upload them.

Everything works, however I noticed it uploads all files and overwrites existing files on the bucket even if they weren't modified since last backup, I only want to upload the delta between each backup.

I know aws cli has the command aws s3 sync <dir> <bucket> which does exactly what I need, however I couldn't find anything equivalent on aws-sdk documentation.

Appreciate the help, thank you!

Answer 1

There is no such feature in aws-sdk. You could instrument it yourself for each file to check the hash of both objects before upload. Or use a community solution https://www.npmjs.com/package/s3-sync-client