Reducing Rails Queries (N+1?)

Question

So in my past application, I was somewhat familiar with using .includes in Rails, but for some reason I'm having a bit of a difficult time in my current scenario.

Here's what I'm working with:

    # If non-existent, create. Otherwise, update.
    existing_data = Page.all
    updated_data = {}
    new_records = []
    @latest_page_data.each do |key, value|
      existing_record = existing_data.find_by(symbol: key)
      if existing_record != nil
        updated_data[existing_record.id] = value
      else
        new_records << Page.new(value)
      end
    end

    if !new_records.empty?
      Page.import new_reocrds
    end
    if !updated_data.empty?
      Page.update(updated_data.keys, updated_data.values)
    end
  end

The problem that I'm having is that the .find_by portion of the code results in a query every single iteration of @latest_page_data . I guess I would think that existing_data would hold all of the data it needs in memory, but obviously it doesn't work that way.

So next, I tried something like this:

# If non-existent, create. Otherwise, update.
existing_data = Page.includes(:id, :symbol)
updated_data = {}
new_records = []
@latest_currency_data.each do |key, value|
  existing_record = existing_data.find_by(symbol: key)

but then rails throws an error, stating:

ActiveRecord::AssociationNotFoundError (Association named 'id' was not found on Page; perhaps you misspelled it?):

so I can't use this example to find the id and symbol attributes.

I tried to take out :id in the Page.includes method, but I need to be able to get to the ID attribute in order to update the respective record later down in the code.

I've also saw some other posts pertaining to this topic, but I think the problem I may be running into is I'm not dealing with associations (and I believe that's what .includes is for? If this is the case, is there any other way that I can reduce all of the queries that I'm submitting here?

Answer 1

The includes method is used to preload associated models. I think what you are looking for is a select . Modifying your code to use select , do this :

existing_data = Page.select(:id, :symbol).load
updated_data = {}
new_records = []
@latest_currency_data.each do |key, value|
  existing_record = existing_data.find_by(symbol: key)
  if existing_record
    updated_data[existing_record.id] = value
  else
    new_records << Page.new(value)
  end
end

The drawbacks of using select over pluck is that since Rails constructs an object for you, so it is slower than a pluck . Benchmark: pluck vs select

Answer 2

Rather than trying to figure out a way to do it in Rails (since I'm not familiar with the 100% correct/accurate Rails way), I just decided to use .pluck and convert it into a hash to get the data that I'm looking for:

existing_data = Page.pluck(:id, :symbol)
existing_data = Hash[*existing_data.flatten]
updated_data = {}
new_records = []
@latest_currency_data.each do |key, value|
  if existing_data.values.include? key
    id = existing_data.find{|k,v| v.include? key}[0]
    updated_data[id] = value
  else
    new_records << Page.new(value)
  end
end

If anyone has a better way, it'd be gladly appreciated. Thanks!