Rails导入大型Excel工作表/ Google工作表

Question

I am currently working on two projects.

One has an excel file with size of about 130mb and you can image how much records it would be containing.

Other is using google sheets with records 25k+ and these will increase over times.

So for such mega uploads how should I go about in rails. I am not finding a detailed tutorial addressing this issue, if someone has it then please share it with me.

Kindly advise me a strategy/gems that I should prefer.

Thanks.

Answer 1

Have you concidered converting to CSV and then importing? There's a tutorial and Gem for that: RailsCasts 396

Answer 2

First, export to CSV. Then split into smaller files, fow example with

split data.csv

(OS X/Linux)

I'd implement the import as a rake task. You could also just generate seed.rb with a bit of string manipulation.

Rails shouldn't have a problem with a 170MB file, but it's often annoying to wait for long-running tasks to finish, especially if you're still debugging.

Alternatively, you can probably import it much faster if you talk directly to mysql. But you'll loose the convenience of rails and should at least do a

Data.all.each do |datum|
   datum.touch
   datum.save!
end

to verify.