在 Heroku 上运行长轨道控制台或 rake 任务

Question

My rails application is hosted on Heroku. I'm trying to process data in my postgresql DB with a straightforward loop

  DocumentText.all.each do |t|
    if is_number? t.src
      puts t.src
      t.targ = t.src
      t.save
    end
  end

The problem I have is my rails console will exit after running through some of my records (not all of them.) I have tried this as a rake task but same problem.

Why is this happening and what are my options?

Answer 1

In your code snippet, you are running n update queries to the database which will take a lot of time and this is why your task is not fully running.

I would suggest running a single update query to the database. For that, add the following gem in your Gemfile

gem 'activerecord-import'

documents = []
DocumentText.all.each do |t|
  documents << t if is_number?(t.src)
end


# MySQL version
DocumentText.import(documents, on_duplicate_key_update: {targ: :src})

# PostgreSQL version (no shorthand version)
DocumentText.import([book], on_duplicate_key_update: {
  conflict_target: [:id], columns: {targ: :src}
})

This should do the trick. If you have data in millions, then I would also recommend running this process in batches. Let me know if you need help with that.

Answer 2

@Pradeep Agrawal's answer lead me to the solution using activerecord-import gem (his answer was copy and pasted from the gem and contained bugs).

documents = []
DocumentText.all.each do |t|
  if is_number?(t.src)
    documents << t
  end
end

DocumentText.import(documents, on_duplicate_key_update: {
  conflict_target: [:id], columns: {targ: :src}
})