Lucene Analyzer tokenizer for substring search

Question

I need a Lucene Tokenizer that can do the following. Given the string "wines bottle caps", the following queries should succeed

• wine
• bott
• cap
• ottl
• aps
• wine bottl

Here is what I have so far. How might I modify it to work? No query less than three characters should work.

public class PorterAnalyzer extends Analyzer {

  private final Version version;

  public PorterAnalyzer(Version version) {
    this.version = version;
  }

  @Override
  @SuppressWarnings("resource")
  protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
    final StandardTokenizer src = new StandardTokenizer(reader);
    TokenStream tok = new StandardFilter(src);
    tok = new LowerCaseFilter( tok);
    tok = new StopFilter( tok, StandardAnalyzer.STOP_WORDS_SET);
    tok = new PorterStemFilter(tok);
    return new TokenStreamComponents(src, tok);
  }

}

Answer 1

I think you are searching for NGramTokenFilter .

Try, for example:

tok=new NGramTokenFilter(tok,2,5);