如何从另一个站点将远程映像下载到Ruby on Rails中的file_column？

Question

first question, hopefully I don't mess it up :)

A bit of a Ruby on Rails newbie (also Ruby newbie) and have stumbled upon a problem with the intended behavior of the application.

I have a file_column :image in model picture that belongs to model product, which can have many pictures.

The file_column works just fine when used as I think it's meant to be used and that's for uploading image using <%= file_column_field "picture", "image" %> etc. That part works just fine.

The problem comes with the intention of having a text field where user can enter a css -selector for an image tag on their site (they've registered the site and the path to the page where the image should be). I haven't been able to figure out how to properly download the image from that other site "under the hood".

Using these two methods both result in Do not know how to handle a string with value 'GIF89ad..... followed by loads of "binary".

Method 1:

url = URI.parse(picture_www.external_url)
Net::HTTP.start(url.host, url.port) {|http|
  resp = http.get(url.path)
  picture_www.image = resp.body unless resp.nil?
}

Method 2:

res = open(picture_www.external_url)
picture_www.image = res.read unless res.nil?

The external_url contains the correct url and the download goes ok, so the problem seems to be in the way I'm trying to assign the image to the file_column field. Naturally the problem could be the way I'm downloading the image, I have no idea TBH where the problem actually lies... :)

Anyone able to help me please?

Update:

Trying to use a tempfile "causes undefined method 'original_filename' for" etc

  Net::HTTP.start(url.host, url.port) {|http|
    resp = http.get(url.path)
    tempfile = Tempfile.new('test.jpg')
    File.open(tempfile.path, 'wb') do |f|
      f.write resp.body
    end
    picture_www.image = tempfile unless resp.nil?
  }

Update2:

Debugging shows me that an uploaded file has attributes @content_type ("image/jpeg" for instance) and @original_path (file name without path) under @_dc_obj and @tmpfile when the tempfile I created does not. Setting these properly would perhaps make this work? How do I set those properly? And if setting those values properly, would the file downloading be done "properly"? After ofcourse re-structuring the code once I get a working solution.

Update3:

From Minver's answer I got the solution for "original_filename" issue and this code seems to work:

  io = open(picture_www.external_url)
  def io.original_filename; base_uri.path.split('/').last; end
  io.original_filename.blank? ? nil : io
  picture_www.image = io

No idea though, if this is the "proper" way to do this or not, but this is what I'll be using for now unless some "clearly the right way to do it" solution appears :)

-Pkauko

Answer 1

The UrlUpload method by Joe Martinez is a good solution but the code is missing a key method. If you over-ride the method_missing , you should always also over-ride the respond_to? method as well. In this case it is especially important since some software uses respond_to? when deciding whether to do a multipart-post.

For example, the Faraday gem does this:

def has_multipart?(body)
  body.values.each do |v|
    if v.respond_to?(:content_type)
      return true
    elsif v.respond_to?(:values)
      return true if has_multipart?(v)
    end
  end
  false
end

So, if you are going to use the UrlUpload code above, I suggest you add the following method:

  def respond_to?(symbol)
    attachment_data.respond_to?(symbol) || super
  end

Then Faraday and other related gems will be able to use an instance of this class to generate a proper multipart-post.

Answer 2

Here ya go

require 'open-uri'

class UrlUpload
  EXTENSIONS = {
    "image/jpeg" => ["jpg", "jpeg", "jpe"],
    "image/gif" => ["gif"],
    "image/png" => ["png"]
  }
  attr_reader :original_filename, :attachment_data
  def initialize(url)
    @attachment_data = open(url)
    @original_filename = determine_filename
  end

  # Pass things like size, content_type, path on to the downloaded file
  def method_missing(symbol, *args)
    if self.attachment_data.respond_to? symbol
      self.attachment_data.send symbol, *args
    else
      super
    end
  end

  private
    def determine_filename
      # Grab the path - even though it could be a script and not an actual file
      path = self.attachment_data.base_uri.path
      # Get the filename from the path, make it lowercase to handle those
      # crazy Win32 servers with all-caps extensions
      filename = File.basename(path).downcase
      # If the file extension doesn't match the content type, add it to the end, changing any existing .'s to _
      filename = [filename.gsub(/\./, "_"), EXTENSIONS[self.content_type].first].join(".") unless EXTENSIONS[self.content_type].any? {|ext| filename.ends_with?("." + ext) }
      # Return the result
      filename
    end
end

# Make it always write to tempfiles, never StringIO
OpenURI::Buffer.module_eval {
  remove_const :StringMax
  const_set :StringMax, 0
}

Answer 3

I don't know but maybe this is what you are looking for. When you save the image you provide a css_selector and gets a image file in return.

This is the view:

<%= form_for(@image) do |f| %>

  <div class="field">
    <%= f.label :css_selector %><br />
    <%= f.text_field :css_selector %>
  </div>

  <div class="actions">
    <%= f.submit %>
  </div>

<% end %>

and this is the model:

class Picture < ActiveRecord::Base

  require 'open-uri' # Required to download the photo
  require 'mechanize' # Good gem to parse html pages

  belongs_to :product

  # Define the css_selector (not required as a filed in the database)
  attr_accessor :css_selector

  # Before we save the image, we download the photo if image has a css_selector value    
  before_save :download_remote_photo, :if => :css_selector_provided?

  private

    # Check if the attribute is provided      
    def css_selector_provided?
      !self.css_selector.blank?
    end

    # This method opens the page where the photo is
    # and grab the url to the image using a css-selector
    def fetch_photo_url
      agent = Mechanize::new
      page = agent.get(HERE_IS_THE_URL_TO_THE_PAGE_YOU_WANNA_SCRAPE)
      doc = Nokogiri::HTML(page.body)

      image_element = doc.at_css(self.css_selector) # Get the image on that page using the css selector
      image_url = image_element[:src]
    end

    def download_remote_photo
      self.image = do_download_remote_photo(fetch_photo_url)
    end

    def do_download_remote_photo(photo_url)
      io = open(URI.parse(URI.escape(photo_url)))
      def io.original_filename; base_uri.path.split('/').last; end
      io.original_filename.blank? ? nil : io
      rescue # catch url errors with validations instead of exceptions (Errno::ENOENT, OpenURI::HTTPError, etc...)
    end

end

Haven't tested the code but I hope you get the idea!