What is involved with changing attachment_fu's storage scheme?

Question

I have a rails application that is using attachment_fu. Currently, it is using :file_system for storage, but I want to change it to :s3 , to allow for better scaling as more files get uploaded.

What is involved with this? I imagine that if I just switch the code to use :s3 , all the old links will be broken. Do I need to just copy the existing files from the file system to S3? A google search hasn't turned up much on the topic.

I would prefer to move the existing files over to S3, so everything is in the same place, but if necessary, the old files can stay where they are, as long as new ones go to S3.

EDIT: So, it is not as simple as copying over the files to S3; the URLs are created using a different scheme. When they are stored in :file_system , the files end up in places like /public/photos/0000/0001/file.name, but the same file in :s3 might end up in 0/1/file.name. I think it is using the id something, and just padding it (or not) with zeros, but I'm not sure of that.

Answer 1

That's correct. The ids are padded using :file_system storage. Instead of renaming all your files, you can alter the s3 backend module to use padded numbers as well.

Copy the partitioned_path method from file_system_backend.rb and put it in s3_backend.rb .

    def partitioned_path(*args)
      if respond_to?(:attachment_options) && attachment_options[:partition] == false
        args
      elsif attachment_options[:uuid_primary_key]
        # Primary key is a 128-bit UUID in hex format. Split it into 2 components.
        path_id = attachment_path_id.to_s
        component1 = path_id[0..15] || "-"
        component2 = path_id[16..-1] || "-"
        [component1, component2] + args
      else
        path_id = attachment_path_id
        if path_id.is_a?(Integer)
          # Primary key is an integer. Split it after padding it with 0.
          ("%08d" % path_id).scan(/..../) + args
        else
          # Primary key is a String. Hash it, then split it into 4 components.
          hash = Digest::SHA512.hexdigest(path_id.to_s)
          [hash[0..31], hash[32..63], hash[64..95], hash[96..127]] + args
        end
      end
    end

Modify s3_backend.rb 's full_filename method to use the partitioned_path .

    def full_filename(thumbnail = nil)
      File.join(base_path, *partitioned_path(thumbnail_name_for(thumbnail)))
    end

attachment_fu will now create paths with the same names as it did with the file_system backend, so you can just copy your files over to s3 without renaming everything.

Answer 2

In addition to nilbus' answer, I had to modify s3_backend.rb 's base_path method to return an empty string, otherwise it would insert the attachment_path_id twice:

def base_path
  return ''
end

Answer 3

What worked for me, in addition to nilbus's answer, was to modify s3_backend.rb's base_path method to still use the path_prefix (which is by default the table name):

def base_path
  attachment_options[:path_prefix]
end

And also, I had to take the attachment_path_id from file_system_backend.rb and replace the one in s3_backend.rb, since otherwise partitioned_path always thought my Primary Key was a String:

def attachment_path_id
  ((respond_to?(:parent_id) && parent_id) || id) || 0
end

Answer 4

Thanks for all those responses which helped a lot. It worked for me too but I had to do this in order to have the :thumbnail_class option working :

def full_filename(thumbnail = nil)
  prefix = (thumbnail ? thumbnail_class : self).attachment_options[:path_prefix].to_s
  File.join(prefix, *partitioned_path(thumbnail_name_for(thumbnail)))
end