How to drop a particular column in csv using ruby 1.9.2

Question

I trying to export the csv to a database. CSV contains some unwanted data which I dont want to store in the database. I need to skip particular columns from the csv which I get and store rest of the data in my database. How to remove particular column from the csv programatically before I push data to the DB. I am using ruby 1.9.2.

Kindly help me out

def csv_import 
    #@parsed_file = csv.open(params[:dump][:file])
    puts "before CSV Reader"

    file = params["dump"]["file"]
    directory = "#{Rails.root.to_s}/public/dump"
      # create the file path
      path = File.join(directory, "#{file.original_filename}")
      # write the file
      File.open(path, "wb") { |f| f.write(file.read) }

     @parsed_file=CSV.open(path, "r")
     @parsed_file.drop(1).each do |row|



     n=0

     c=ModelName.new
     c.invoiceno=row[2]
     c.invoice_date=row[3]
     c.orderrefno = row[4]
     c.skucode = row[7]
     c.quantiy = row[8]
     c.amount = row[9]
     c.trackno=row[11]
     c.dispatched = "No"
     c.mailsenttoc = "No"
     c.mailsenttobluedart = "No"

     if c.save
        n=n+1
        GC.start if n%50==0
     end
   end 
end

As you can see I have skipped couple of columns like 1,5,6,10

Answer 1

Not sure if this helps, but you could also use remote_table :

require 'remote_table'
def csv_import
  # [...]
  RemoteTable.new("file://#{path}", :format => :csv, :headers => :false).each do |row|
    c = ModelName.new
    c.invoiceno = row[2]
    c.invoice_date = row[3]
    c.orderrefno = row[4]
    c.skucode = row[7]
    c.quantiy = row[8]
    c.amount = row[9]
    c.trackno = row[11]
    c.dispatched = "No"
    c.mailsenttoc = "No"
    c.mailsenttobluedart = "No"
    if c.save
      # [...]
    end
  end
  # [...]
end

Answer 2

You can use activewarehouse-etl to do this.

https://github.com/activewarehouse/activewarehouse-etl

It will allow you to specify the columns that you want to pull in from the csv file and then will bulk upload it to your database.

You can also use it to clean up and verify the data that you are putting in as well as set default values.