Use CDN with carrierwave + fog in s3 + cloudfront with rails 3.1

Question

I'm using fog with carrierwave in my website. But the images load very very slowly.

Then I want to speed up loading of images with a CDN.

I have followed this tutorial for create the CDN for images:

http://maketecheasier.com/configure-amazon-s3-as-a-content-delivery-network/2011/06/25

I have now my distribution deployed for images but I don't know how works fine the cdn. I have in initializers/fog.rb the next configuration:

CarrierWave.configure do |config|
  config.fog_credentials = {
    :provider               => 'AWS',
    :aws_access_key_id      => 'key',
    :aws_secret_access_key  => 'key',
    :region                 => 'eu-west-1'
  }
  config.fog_host = "http://da33ii2cvf53u.cloudfront.net" #config.asset_host instead of config.fog_host for new fog gem versions
  config.fog_directory  = 'pin-pro'
  config.fog_public     = false
  #config.fog_attributes = {'Cache-Control' => 'max-age=315576000'} 
end 

I dont know if this is correct, but in my local machine it does not works fine for me. I see the image location, is the same route as before:

https://s3-eu-west-1.amazonaws.com/pin-pro/uploads/pins/medium_610cafbe-5d43-4223-ab0e-daa4990863c4.jpg?AWSAccessKeyId=AKIAIDX34WHYKB3ZKFVA&Signature=RwQriNpiRXaTxyfYVvYjsvclUa8%3D&Expires=1333203059

How can I add a CDN to fog file in carrierwave with s3 and cloudfront?

Answer 1

It looks like you haven't added the line below to your config. You will need to replace the sample address below with your cloudfront address from Amazon.

From the github README: https://github.com/jnicklas/carrierwave

"You can optionally include your CDN host name in the configuration. This is highly recommended, as without it every request requires a lookup of this information"

config.asset_host = "http://c000000.cdn.rackspacecloud.com"

Answer 2

CarrierWave won't work when you set config.fog_public = false and point config.asset_host to a CloudFront distribution. This has been documented multiple times:

https://github.com/carrierwaveuploader/carrierwave/issues/1158 https://github.com/carrierwaveuploader/carrierwave/issues/1215

In a recent project I was happy using CarrierWave to handle uploads to S3, but wanted it to return a signed CloudFront URL when using Model.attribute_url . I came up with the following (admittedly ugly) workaround that I hope others can benefit from or improve upon:

Add the 'cloudfront-signer' gem to your project and configure it per the instructions. Then add the following override of /lib/carrierwave/uploader/url.rb in a new file in config/initializers (note the multiple insertions of AWS::CF::Signer.sign_url ):

module CarrierWave
      module Uploader
        module Url
          extend ActiveSupport::Concern
          include CarrierWave::Uploader::Configuration
          include CarrierWave::Utilities::Uri

          ##
          # === Parameters
          #
          # [Hash] optional, the query params (only AWS)
          #
          # === Returns
          #
          # [String] the location where this file is accessible via a url
          #
          def url(options = {})
            if file.respond_to?(:url) and not file.url.blank?
              file.method(:url).arity == 0 ? AWS::CF::Signer.sign_url(file.url) : AWS::CF::Signer.sign_url(file.url(options))
            elsif file.respond_to?(:path)
              path = encode_path(file.path.gsub(File.expand_path(root), ''))

              if host = asset_host
                if host.respond_to? :call
                  AWS::CF::Signer.sign_url("#{host.call(file)}#{path}")
                else
                  AWS::CF::Signer.sign_url("#{host}#{path}")
                end
              else
                AWS::CF::Signer.sign_url((base_path || "") + path)
              end
            end
          end

        end # Url
     end # Uploader
end # CarrierWave

Then override /lib/carrierwave/storage/fog.rb by adding the following to the bottom of the same file:

require "fog"

module CarrierWave
  module Storage
    class Fog < Abstract
       class File
          include CarrierWave::Utilities::Uri
          def url
             # Delete 'if statement' related to fog_public
             public_url
          end
       end
    end
  end
end

Lastly, in config/initializers/carrierwave.rb :

config.asset_host = " http://d12345678.cloudfront.net "

config.fog_public = false

That's it. You can now use Model.attribute_url and it will return a signed CloudFront URL to a private file uploaded by CarrierWave to your S3 bucket.

Answer 3

似乎亚马逊 cdn 不适用于config.fog_public = false ，所以私有文件只能从 s3 访问，不能从 cdn

Answer 4

After some searching and struggling with this for a long time I found a page that says that CarrierWave doesn't support CloudFront signed urls. CloudFront signed urls are different than S3 signed urls, which caused me some confusion. Once I figured that out, it was a lot easier to know what to do.

If you configure CarrierWave with config.fog_public = false then it will automatically begin signing S3 urls, but it can't be configured to work with Fog and CloudFront private content in the version of CarrierWave I'm using (1.0.0) . I even tried using the carrierwave-aws gem and that didn't help either.

So what would happen is that CarrierWave would sign the URL and the host would look something like this:

https://my_bucket_name.s3-us-west-2.amazonaws.com/uploads/...?signature...

That points directly to the S3 bucket, but I needed it to point to CloudFront. I needed the host to look like this:

https://s3.cloudfront_domain_name.com/uploads/...

And what would happen if I set config.asset_host equal to my CloudFront location is I'd get this, (with double slashes before "uploads"):

https://s3.cloudfront_domain_name.com//uploads/...

That, too, made it clear CarrierWave wasn't yet designed to be used with CloudFront. Hopefully they'll improve it. This was my work-around. It's ugly, but it worked to get done what I needed without needing to modify CarrierWave itself, as I hope CarrierWave will at some point add support for CloudFront.

1. First I did a regex find/replace on my url and removed the S3 host portion and put on my CloudFront host portion. cf_url = s3_url.gsub("my_bucket_name.s3-us-west-2.amazonaws.com", "s3.cloudfront_domain_name.com")
2. Next I did another regex find/replace to remove the S3 signed url at the end of the string: non_signed_cf_url = cf_url.gsub(/\\?.+/, '') This is because the signature will be incorrect because it was using the API for S3 and not for CloudFront for signing the URL.
3. Now I re-sign the URL myself, using the cloudfront-signer gem: signed_cf_url = Aws::CF::Signer.sign_url(non_signed_cf_url, :expires => 1.day.from_now)

There are a few other things you need to be aware of when serving private content on CloudFront:

• In the Cache Behavior Settings for your path pattern (not necessarily the default one), set: "Restrict Viewer Access (Use Signed URLs or Signed Cookies)" to "Yes"
• Set "Trusted Signers" to "self"
• Set "Query String Forwarding and Caching" to "Forward all, cache based on all" if you want to use other query strings more than the CloudFront signature in your url, such as response-content-disposition and response-content-type (I was able to get these to work successfully, but they have to be url_encoded properly.)
• In your CloudFront Origin Settings, set your access-identity and set "Grant Read Permissions on Bucket" to "Yes, Update Bucket Policy"
• In your General Distribution Settings, make sure "Distribution State" is "Enabled" and that you've added a CNAME to "Alternate Domain Names (CNAMEs)" if you're using one.
• If using a CNAME, make sure your DNS is correctly configured to point it to your CloudFront distribution's name.
• Lastly, once you set the configurations there is a long wait while AWS updates the distribution, so you won't see your changes happen right away. It may seem like your app/website is still broken until the changes propagate through CloudFront. This can make configuring it difficult because if you get it wrong you have to wait a long time before you can see your changes take effect and you may not be sure what happened. But with these settings I was able to get it working for me.
• You can also create more than one caching path pattern so that some content is private and requires a CloudFront signed url, and other content isn't. For example, I set a path pattern of *.mp4 that requires a signature for all mp4 files and placed that above the default behavior. And then I have the default cache behavior set to NOT require signed urls, which allows all other files - such as images - to be publicly accessible through the CloudFront distribution.

Answer 5

As mentioned before Fog or carrierwave-aws don´t allow to set private S3 Bucket and public access from a Cloudfront distribution. In order to do this you need to override the url method in your uploader like this:

def url(*args)
  if file.respond_to?(:url) and not file.url.blank?
    if args
      "https://*my_cloudfront_url*/#{store_dir}/#{args.join('_').to_s}_#{identifier}"
    else
      "https://*my_cloudfront_url*/#{store_dir}/#{identifier}"
    end
  elsif current_path
    File.expand_path(current_path).gsub(File.expand_path(public), '')
  end
end