Rails: Best way to allow users to upload images to either a Dropbox linked folder or “our” storage on Amazon S3

Question

I am working on a project where the user joins a "stream". During stream setup, the person who is creating the stream (the stream creator) can choose to either:

1. Upload all photos added to the stream by members to our hosting solution (S3)
2. Upload all photos added to the stream by members to the stream creator's own Dropbox authenticated folder

In the future I would like to add more storage providers (such as Drive, Onesky etc)

There is a couple of different questions I have in regards to how to solve this.

1. What should the structure be in the database for photos? I currently only have photo_url, but that won't be easy to manage from a data perspective with pre-signed urls and when there are different ways a photo can be uploaded (s3, dropbox etc.)
2. How should the access tokens for each storage provider be stored? Remember that only the stream creator's access_token will be stored and everyone who is on the stream will share that token when uploading photos
3. I will add iOS and web clients in the future that will do a direct upload to the storage provider and bypass the server to avoid a heavy load on the server

Answer 1

As far as database storage, your application should dictate the structure based on the interface that you present both to the user and to the stream. If you have users upload a photo and they don't get to choose the URI, and you don't have any hierarchy within a stream, then I'd recommend storing just an ID and a stream_id in your main photo table.

So at a minimum you might have something looking like

create table photos(id integer primary key, stream_id integer references streams(id) not null);

But you probably also want description and other information that is independent of storage.

The streams table would have all the generic information about a stream, but would have a polymorphic association to a class dependent on the type of stream. So you could use that association to get an instance of S3Stream or DropBoxStream based on what actual stream was used. That instance (also an ActiveRecord resource) could store the access key, and for things like dropbox, the path to the folder etc. In addition, that instance could provide methods to construct a URI given your Photo object. If a particular technology needs to cache signed URIs, then say the S3Stream object could reference a S3SignedUrl model where the URIs are signed. If it turns out that the signed URL code is similar between DropBox and S3, then perhaps you have a single SignedUrl model.

When you design the ios and android clients, it is critical that they are not given access to the stream owner's access tokens. Instead, you'll need to do all the signing inside your server app. You wouldn't want a compromise of a device to lead to exposing the access token creating billing problems as well as privacy exposures. Hope this helps.

Answer 2

we setup a lot of rails applications with different kind of file storages behind it.

1. Yes, just an url is not manageable in the future. To save a lot of time you could use gems like carrierwave or paperclip . They handle all the thumbnail generation and file validation. One approach is, that you could upload the file from the client directly to S3 or Dropbox to a tmp folder and just tell your Rails App "Hey, here is the url of a new upload file" and paperclip and carrierwave will take care of the thumbnail generation and storaging. ( Example for paperclip )

2. Don't know exactly how your stream works, so I cannot give a good answer to this -.-

3. With the setup I mentioned in 1. you should upload form your different clients directly to S3 or Dropbox etc. and after uploading, the client tells the Rails Backend that it should import the file from that url. (And before paperclip or carrierwave finish their processing you could use the tmp url from the file to display something directly in your stream)