Solr: NullPointerException on Spellcheck (FSTCompletionLookup)

Question

I posted this on the solr-user mailing list, but haven't heard feedback , so FBOW I'm trying SO. Sorry for the cross-posting.

Problem

Solr's 'suggest/spellcheck' component throws a NullPointerException (NPE) when a user executes a search. It fails on FSTCompletionLookup (line 244)

Update this was a configuration error.

In my haste/carelessness, instead of defining separate "spellcheck" and "suggest" components, I defined only "suggest". (More specifically I copied over the ch10 examples from "solr in action", but did not copy the "spellcheck" component." )

When solr complained about not finding 'spellcheck' component, I looked over my (bad) solrconfig.xml and thought "hmm. 'spellcheck..component probably should be 'suggest'". It worked after re-indexing..and appeared to function correctly.

Cracking open the hard copy, sitting down in the easy chair, looking carefully over the chapter brought the issue to my attention.

Question

• How can I get past this? Cache Warming? Term Vectors?

Context

• Solr 4.4. ( I'm using 4.4 to match "what's in production." I could upgrade to the latest if necessary. )
• Basically I'm applying the examples from Solr in Action and my configuration matches the example.
• Inconsistent results: If I reindex the site, then run a particular search, it succeeds. However if I restart Solr, the same query fails with an NPE

Dictionary Details

The Suggester uses the 'currently open searcher' to build it's dictionary. This might explain the 'inconsistency above'[ search after fresh restart gives a NPE, long-running instance succeeds]

  reader = searcher.getIndexReader();
  dictionary = new HighFrequencyDictionary(reader, field, threshold);

Request Handler Configuration

  <str name="spellcheck">on</str>
  <str name="spellcheck.dictionary">suggestDictionary</str>        
  <str name="spellcheck.extendedResults">false</str>
  <str name="spellcheck.count">5</str>
  <str name="spellcheck.alternativeTermCount">2</str>
  <str name="spellcheck.maxResultsForSuggest">5</str>
  <str name="spellcheck.collate">true</str>
  <str name="spellcheck.collateExtendedResults">true</str>
  <str name="spellcheck.maxCollationTries">5</str>
  <str name="spellcheck.maxCollations">3</str>

Spellcheck component

<searchComponent class="solr.SpellCheckComponent" name="suggest">
    <lst name="spellchecker">
        <str name="name">suggestDictionary</str>
        <str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
        <str name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookupFactory</str>
        <str name="field">title</str>           
     <!--   <str name="field">suggest</str> -->
        <float name="threshold">0.</float>
        <str name="buildOnCommit">true</str>
    </lst>
</searchComponent>

Field type definition

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <!-- Use EdgeNGramFilter for wildcard search -->
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
    <!-- in this example, we will only use synonyms at query time
    <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
    -->
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

field definition

thanks in advance

Answer 1

I was also having a NPE issue with FSTCompletionLookup suggest component, could not remember the exact line number of MPE cause, i figured out ( with my config and solr were using custom implementation of FSTCompletionLookup) that this was because the FSTCompletionLookup suggest dictionary was not created on startup, but this was available after reload of core after startup.

{solr-version} : 4.7.x+

I added the below configuration which build suggest dic on startup, hope this helps

<query>
    <listener event="firstSearcher" class="solr.QuerySenderListener">
      <arr name="queries">
        <lst> <str name="qt">/suggest</str>
                <str name="spellcheck.build">true</str>
        </lst>
      </arr>
    </listener>

</query>