Unit Testing Tire (Elastic Search) - Filtering Results with Method from to_indexed_json

Question

I am testing my Tire / ElasticSearch queries and am having a problem with a custom method I'm including in to_indexed_json. For some reason, it doesn't look like it's getting indexed properly - or at least I cannot filter with it.

In my development environment, my filters and facets work fine and I am get the expected results. However in my tests, I continuously see zero results.. I cannot figure out where I'm going wrong.

I have the following:

def to_indexed_json
 to_json methods: [:user_tags, :location_users]
end

For which my user_tags method looks as follows:

def user_tags
  tags.map(&:content) if tags.present?
end

Tags is a polymorphic relationship with my user model:

has_many :tags, :as => :tagable

My search block looks like this:

def self.online_sales(params)
  s = Tire.search('users') { query { string '*' }}
    filter = []
    filter << { :range => { :created_at => { :from => params[:start], :to => params[:end] } } }
    filter << { :terms => { :user_tags => ['online'] }}
    s.facet('online_sales') do
      date :created_at, interval: 'day'
      facet_filter :and, filter
    end
  end
end

I have checked the user_tags are included using User.last.to_indexed_json :

{"id":2,"username":"testusername", ... "user_tags":["online"] }

In my development environment, if I run the following query, I get a per day list of online sales for my users:

@sales = User.online_sales(start_date: Date.today - 100.days).results.facets["online_sales"]


"_type"=>"date_histogram", "entries"=>[{"time"=>1350950400000, "count"=>1, "min"=>6.0, "max"=>6.0, "total"=>6.0, "total_count"=>1, "mean"=>6.0}, {"time"=>1361836800000, "count"=>7, "min"=>3.0, "max"=>9.0, "total"=>39.0, "total_count"=>7, "mean"=>#<BigDecimal:7fabc07348f8,'0.5571428571 428571E1',27(27)>}....

In my unit tests, I get zero results unless I remove the facet filter..

{"online_sales"=>{"_type"=>"date_histogram", "entries"=>[]}}

My test looks like this:

it "should test the online sales facets", focus: true do
  User.index.delete
  User.create_elasticsearch_index
  user = User.create(username: 'testusername', value: 'pass', location_id: @location.id)  
  user.tags.create content: 'online'  
  user.tags.first.content.should eq 'online'
  user.index.refresh
  ws = User.online_sales(start: (Date.today - 10.days), :end => Date.today) 
  puts ws.results.facets["online_sales"]
end

Is there something I'm missing, doing wrong or have just misunderstood to get this to pass? Thanks in advance.

-- EDIT --

It appears to be something to do with the tags relationship. I have another method, ** location_users ** which is a has_many through relationship. This is updated on index using:

def location_users
  location.users.map(&:id)
end

I can see an array of location_users in the results when searching. Doesn't make sense to me why the other polymorphic relationship wouldn't work..

-- EDIT 2 --

I have fixed this by putting this in my test:

User.index.import User.all
sleep 1

Which is silly. And, I don't really understand why this works. Why?!

Answer 1

Elastic search by default updates it's indexes once per second.

This is a performance thing because committing your changes to Lucene (which ES uses under the hood) can be quite an expensive operation.

If you need it to update immediately include refresh=true in the URL when inserting documents. You normally don't want this since committing every time when inserting lots of documents is expensive, but unit testing is one of those cases where you do want to use it.

From the documentation:

refresh

To refresh the index immediately after the operation occurs, so that the document appears in search results immediately, the refresh parameter can be set to true. Setting this option to true should ONLY be done after careful thought and verification that it does not lead to poor performance, both from an indexing and a search standpoint. Note, getting a document using the get API is completely realtime.