Is it possible to updade only a piece of a file that was already uploaded to S3?

Question

Imagine I did upload a 50mb file to S3.

After uploading it completely, the file was changed, by adding 5mb of data to it. Now the file is 55mb.

Is it possible for me to upload only these 5mb that were added to S3 or do I always have to re-upload the whole file?

Possibly something like a PUT method that would accept a content-range so I could update just this specific range of bytes.

Answer 1

Currently S3 API doesn't allow you to upload changed parts (ie to patch the file). You need to reupload the whole file.

Another option could be to use your own file format to keep file as a set of chunks. But this makes it incompatible with other s3 clients.

Answer 2

You might need to look into something like rsync if this is something you're going to do regularly. Rsync keeps track of changes made to files and keeps them synced between servers. Here at work, we use a wrapper for rsync called duplicity , which allows us to backup files to S3, and only uploads the difference between changed files.

It all depends on what your specific needs are, however.