Managing files on Amazon S3

Question

I have a git repository that stores audio files.

Obviously, it's not the best usage of git, and the repo has become quite large.

As an alternative, I would like to be able to manipulate these audio files at the command line, "commiting" when some work is done.

Is this type of context possible with manipulating Amazon S3 files at the command line?

Or do you scp , for example, files to S3?

Answer 1

There are some rsync tools to S3 that may work for you, here is an example which I have not tried: http://www.s3rsync.com/

How important are the older versions of the audio? Amazon S3 buckets can have 'versioning' turned on, and you get full versioning support. You pay full $ for each version - I don't know if you have 10 GB or 10TB to store, and your budget, etc... The amazon versioning is nice, but there are not a lot of tools that fully support it.

Answer 2

To manipulate S3 files you will first have to download it and then upload it when you are done, this is relatively simple to do.

However, if the amount of files you have is truly large, the slow transfer rate and bandwidth charge will kill you. If you don't have that much files, DropBox is built on top of S3 and have syncing and a rudimentary version control, bandwidth is not charged..

I felt like using a good networked storage system and git on your LAN is still the better idea.