简体   繁体   中英

Why is my sunspot_rails query taking so long? What should i do to speed it up?

I am new to Solr/Sunspot and i'm sure i've done something wrong.

I have sunspot_rails 1.2.1 working with rails 2.3.4 (tried other versions but they wouldn't work and we are hoping not to upgrade rails just now)

The reindex worked and i am getting results, but the search is taking much too long. About 25 seconds to do a fulltext search on two fields with just under 78,000 documents.

I have this:

class Abstract < ActiveRecord::Base
  searchable do
    text :original_title, :headline
  end
end

Searching like this:

Abstract.search do
  fulltext 'test'
  paginate :page => 1, :per_page => 30
end

This is the only clue i have to go on:

Solr Select (25275.2ms) {:fq=>["type:Abstract"], :start=>0, :defType=>"dismax", :q=>" +(test)", :fl=>"* score", :rows=>30, :qf=>"original_title_text headline_text"}

It's not just a one-off first search, this is every time.

If i add more criteria (such as updated_at between two dates) it increases up to about 8 minutes!

I've been asked to note that this is in a development environment.

Update

The problem is not with Solr. If i visit this:

http://localhost:8982/solr/select?indent=on&version=2.2&q=+%2B(test)&fq=type:Abstract&defType=dismax&start=0&rows=10&fl= *%2Cscore&qt=standard&wt=standard&explainOther=&hl.fl=&qf=original_title_text+headline_text

it does the same query and takes a miniscule amount of time (something like 4 milliseconds!)

So why does it take so long to query through Rails? it's either sunspot, or rsolr, or as John suggested, perhaps it's starting up a new instance of Solr, but i don't think that's the case.

Should i try switching to acts_as_solr? I thought that was a bit out of date now.

I'm on sunspot 1.2.1 and rsolr 0.12.1 - would newer versions help here?

Interesting new information

Moving up to Rails 2.3.14 reduced the query time to 18 seconds!

RESULT!

I have now tried sunspot 1.3.0 with RSolr 1.0.6 and finally i am getting the fast response that i was looking for! This required me to move to the stable version of rails 2.3. Shame, we didn't want to upgrade the rails version, but it seems it was necessary.

At Brightbox we've got about 36 million docs stored in Solr in one particular cluster. We store and search the docs using Ruby (though not with sunspot). We store the documents in with the index and it takes about 36gig on disk.

The first search after a restart of solr can take up to a minute or more, but subsequent searches (even very different ones) typically take less than half a second (even searches returning millions of documents and sorting them by another field value).

We've upped the maximum memory heap site to 1gig (-Xmx1024m) and it typically grows up to it almost instantly, but ten is pretty stable. We did tune the schema carefully for our data, but nothing special.

Something is clearly seriously wrong with your setup for it to take so long every time with only 78k documents.

Without much more detail from you about what is going on I'd suggest: first off, upping your max memory heap setting - if yours it too low, then it'll be reading the index off disk every time, and will likely be triggering the garbage collector often (which uses lots of cpu).

Secondly, make sure sunspot isn't doing something odd like starting up a new solr instance on every search. There should be a solr daemon running the whole time.

Thirdly, if you still have a problem, try making the call to solr directly yourself with curl or something. try simplifying the query bit by bit to see if one particular part of it is the problem (maybe lose everything but defType and qf and start adding it back in from there).

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM