Uploading a file using the Ruby SDK to Amazon S3

Question

I am trying to upload a file. A simple hello.txt. I was following the docs, and I am unable to upload it to my bucket.

#  START AWS CLIENT

s3 = Aws::S3::Resource.new
bucket = s3.bucket(BUCKET_NAME)

begin

  s3.buckets[BUCKET_NAME].objects[KEY].write(:file => FILE_NAME)
  puts "Uploading file #{FILE_NAME} to bucket #{BUCKET_NAME}."

  bucket.objects.each do |obj|
    puts "#{obj.key} => #{obj.etag}"
  end

rescue Aws::S3::Errors::ServiceError
  # rescues all errors returned by Amazon Simple Storage Service
end

I was following http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjSingleOpRuby.html

The error:

➜ s3-tester ruby main.rb /Users/.rvm/gems/ruby-2.1.1/gems/aws-sdk-resources-2.0.34/lib/aws-sdk-resources/collection.rb:79:in 'method_missing: undefined method []' for ' <Aws::Resources::Collection:0x000001031e5100> (NoMethodError)' from 'main.rb:18:in <main> '

Answer 1

The primary issue is that you have version 2 of the AWS SDK for Ruby installed, but you are referencing the documentation for version 1 . Version 2 documentation can be found here:

http://docs.aws.amazon.com/sdkforruby/api/index.html

To update your example to use version 2:

s3 = Aws::S3::Resource.new
bucket = s3.bucket(BUCKET_NAME)

begin

  bucket.object(KEY).upload_file(FILENAME)
  puts "Uploading file #{FILE_NAME} to bucket #{BUCKET_NAME}."

  bucket.objects.each do |obj|
    puts "#{obj.key} => #{obj.etag}"
  end

rescue Aws::S3::Errors::ServiceError
    # rescues all errors returned by Amazon Simple Storage Service
end

The primary differences:

• Version 1 used the #[] method on a collection to reference an object by its key. Version 2 has two methods, #objects() and #object(key) . The latter is the getter. The former enumerates all objects in the bucket.
• Version 2 has a specialized #upload_file method that manages upload an object from disk. This is similar to #write from version 1, but it can also use multiple threads to upload large object parts in parallel.

Answer 2

client = Aws::S3::Client.new(region: 'us-east-1')
resource = Aws::S3::Resource.new(client: client)
bucket = resource.bucket(BUCKET_NAME)
begin
  # s3.buckets[BUCKET_NAME].objects[KEY].write(:file => FILE_NAME)
  # puts "Uploading file #{FILE_NAME} to bucket #{BUCKET_NAME}."

  bucket.objects.each do |o|
    puts o.key
  end

rescue Aws::S3::Errors::ServiceError
  # rescues all errors returned by Amazon Simple Storage Service
end

Answer 3

I used a script like the following which would create a new bucket if it does not exist and then upload the chosen file to it.

#!/usr/bin/env ruby
#

require 'rubygems'
require 'aws-sdk'

bucket_name = ARGV[0]
file_name = ARGV[1]


# Get an instance of the S3 interface.
s3 = Aws::S3::Client.new(region: 'us-east-1')

key = File.basename(file_name)
resp = s3.list_buckets()
buckets = resp.data.buckets

if buckets.select { |b| b.name == bucket_name }.length == 0
  puts 'creating bucket'
  s3.create_bucket(bucket: bucket_name)
end

puts "Uploading file #{file_name} to bucket #{bucket_name}..."

# Upload a file.
s3.put_object(
  :bucket => bucket_name,
  :key    => key,
  :body   => IO.read(file_name)
)

if you saved that to upload.rb you could upload simple.txt to my_bucket by running

$ ruby upload.rb my_bucket simple.txt

Answer 4

Short Answer: You should add a method to check if the bucket already exists or not. By doing this you will be following the best practices, and minimizing the risk of failures.

Code would be something like:

client = Aws::S3::Client.new(http_wire_trace: true) // Note: Enable wire_trace only in debug mode.
    
create_bucket_if_missing(client, bucket_name)

client.put_object(
     :bucket => bucket_name,
     :key    => key,
     :body   => File.read(file_path)
   )

def create_bucket_if_missing(client, bucket_name)
  begin
    client.head_bucket(bucket: bucket_name)
    return
  rescue StandardError
    puts "Failed to find bucket #{bucket_name}, creating it"
    client.create_bucket({bucket: bucket_name})
  end
end