Moving to the last line of a file while reading it in a .each loop in Ruby

Question

I'm reading in a file that can contain any number of rows.

I only need to save the first 1000 or so, passed in as a variable " recordsToParse ".

If I reach my 1000 line limit, or whatever it's set to, I need to save the trailer information in the file to verify total_records , total_amount etc.

So, I need a way to move my "pointer" from where ever I am in the file to the last line and run through one more time.

file = File.open(file_name)

parsed_file_rows = Array.new
successful_records, failed_records = 0, 0
file_contract = file_contract['File_Contract']
output_file_name = file_name.gsub(/.TXT|.txt|.dat|.DAT/,'')

file.each do |line|
  line.chomp!
  line_contract = determine_row_type(file_contract, line)

  if line_contract
    parsed_row = parse_row_by_contract(line_contract, line)
    parsed_file_rows << parsed_row
    successful_records += 1
  else
    failed_records += 1
  end

  if (not recordsToParse.nil?)
    if successful_records > recordsToParse
      # move "pointer" to last line and go through loop once more
      #break;
    end
  end

end
store_parsed_file('Parsed_File',"#{output_file_name}_parsed", parsed_file_rows)
[successful_records, failed_records]

Answer 1

Use IO.seek with IO::SEEK_END to move your pointer to the end of the file, then move up to the last CR, then you have your last line.

This would only be worthwhile if the file is very big, otherwise just follow the file.each do |line| to the last line or you could read the last line like this IO.readlines("file.txt")[-1] .

The easiest solution is to use a gem like elif

require "elif"

lastline = Elif.open("bigfile.txt") { |f| f.gets }

It reads your lastline in a snap undoubtedly using seek.

Answer 2

This is one of those times I'd take advantage of the OS's head and tail commands using something like:

head = `head -#{ records_to_parse } #{ file_to_read }`.split("\n")
tail = `tail -1 #{ file_to_read }

head.pop if (head[-1] == tail.chomp)

Then write it all out using something like:

File.open(new_file_to_write, 'w') do |fo|
  fo.puts head, tail
end