How do I read line by line a text file in ruby (hosting it on s3)?

Question

I know I've done this before and found a simple set of code, but I cannot remember or find it:(.

I have a text file of records I want to import into my Rails 3 application.

Each line represents a record. Potentially it may be tab delimited for the attributes, but am fine with just a single value as well.

How do I do this?

Answer 1

File.open("my/file/path", "r").each_line do |line|
  # name: "Angela"    job: "Writer"    ...
  data = line.split(/\t/)
  name, job = data.map{|d| d.split(": ")[1] }.flatten
end

Related topic

What are all the common ways to read a file in Ruby?

Answer 2

You want IO.foreach :

IO.foreach('foo.txt') do |line|
  # process the line of text here
end

Alternatively, if it really is tab-delimited, you might want to use the CSV library:

File.open('foo.txt') do |f|
  CSV.foreach(f, col_sep:"\t") do |csv_row|
    # All parsed for you
  end
end

Answer 3

  IO.foreach("input.txt") do |line| 
    out.puts line
    # You might be able to use split or something to get attributes
    atts = line.split
  end

Answer 4

Have you tried using OpenURI ( http://ruby-doc.org/stdlib-2.1.2/libdoc/open-uri/rdoc/OpenURI.html )? You would have to make your files accessible from S3.

Or try using de aws-sdk gem ( http://aws.amazon.com/sdk-for-ruby ).

Answer 5

You can use OpenURI to read remote or local files.

Assuming that your model has an attachment named file :

# If object is stored in amazon S3, access it through url
file_path = record.file.respond_to?(:s3_object) ? record.file.url : record.file.path
open(file_path) do |file|
  file.each_line do |line|
    # In your case, you can split items using tabs
    line.split("\t").each do |item|
      # Process item
    end
  end
end