Get download stream of S3 Object and pass it to another method in ruby

Question

Amazon S3 API for Ruby only streams objects when passing a block to the read method. I'm developing a driver for em-ftpd, and It needs an IOish object to stream the S3 data to the client. If simply pass myS3Object.read, whole file is loaded into the memory, as stated by S3 API. Is there any way of encapsulating that into something like a custom IO class, so I can pass the stream to em-ftpd?

Here is my code:

def get_file(path)
  #loading whole file into memory - not efficient
  yield bucket.objects[path].read
end

Here is the code inside em-ftpd that gets the result of my code, preferably as an IOish object, using a block to get its data chunks:

# send a file to the client
def cmd_retr(param)
  send_unauthorised and return unless logged_in?
  send_param_required and return if param.nil?

  path = build_path(param)

  @driver.get_file(path) do |data|
    if data
      send_response "150 Data transfer starting #{data.size} bytes"
      send_outofband_data(data)
    else
      send_response "551 file not available"
    end
  end
end

Thank you.

Answer 1

事实证明我需要创建一个具有S3数据缓冲区和读取和eof方法的类。