Read file and validate rows in Ruby

Question

I have a CSV file as following

ID      Required  -- these are headers
SD0005   Yes      -- row information

I have to validate each row against header. Say ID contains letters and numbers and length should not be more than 6. Required header should be either yes or no in every row.

How can I achieve this functionality in Ruby if I have to process a huge file which has more than 1000 rows with good performance?

I'm reading particular row against each header as follows

CSV.foreach('file path', :headers => true) do |csv_obj| 
csv_obj['ID'] 
csv_obj['Required']

Is there a way to know which condition failed while validating column against header for a row. I need to know for which condition it failed and print it out

New to ruby. Appreciate help

Answer 1

To get the data into Ruby from the CSV file, try the following:

# This will read the data in with headers, convert the column names to symbols, 
# and then turn each resulting CSV::Row instance into a hash

data = CSV.read('temp.csv', headers: true, header_converters: :symbol).map(&:to_h)

this should return the following:

=> [{:id=>"SD0005", :required=>" yes"}, ...]

Once you have all of your information in a format you can work with in Ruby, you could make a method that checks the validity of each ID.

def valid_id?(id_string)
  # uses Regular Expressions to ensure ID is 6 
  # characters that consist of only numbers/letters
  # The double-bang(!!) turn a truthy value to `true`
  # or false value to `false`

  !!id_string.match(/^[\D|\d]{6}$/)
end

If you wanted to test another column's validity, do so with a separate method.

def valid_required?(req_column)
  req_column.downcase == 'yes' ||   req_column.downcase == 'no'
end

Create a main validator method

def all_valid?(row)
  valid_id?(row[:id]) && valid_required?(row[:required])
end

Then keep only the records whose ID is valid

# #select keeps values whose block evaluates to `true`
valid_records = data.select { |record| all_valid?(record) }