Elasticsearch，如何将词接ngram呢？

Question

I'd like to concatenate words then ngram it.
What's the correct setting for elasticsearch?

In english,

from: stack overflow

==> stackoverflow : concatenate first,

==> sta / tac / ack / cko / kov / ... and etc (min_gram: 3, max_gram: 10)

Answer 1

To do the concatenation I'm assuming that you just want to remove all spaces from your input data. To do this, you need to implement a pattern_replace char filter that replaces space with nothing.

Setting up the ngram tokenizer should be easy - just specify your token min/max lengths.

It's worth adding a lowercase token filter too - to make searching case insensitive.

curl -XPOST localhost:9200/my_index -d '{
  "index": {
    "analysis": {
        "analyzer": {
            "my_new_analyzer": {
                "filter": [
                    "lowercase"
                ],
                "tokenizer": "my_ngram_tokenizer",
                "char_filter" : ["my_pattern"],
                "type": "custom"
            }
        },
       "char_filter" : {
          "my_pattern":{
            "type":"pattern_replace",
            "pattern":"\u0020",
            "replacement":""
           }
        }, 
        "tokenizer" : {
                "my_ngram_tokenizer" : {
                    "type" : "nGram",
                    "min_gram" : "3",
                    "max_gram" : "10",
                    "token_chars": ["letter", "digit", "punctuation", "symbol"]
                }
            }
    }
  }
}'

testing this:

curl -XGET 'localhost:9200/my_index/_analyze?analyzer=my_new_analyzer&pretty' -d 'stack overflow'

gives the following (just a small part shown below):

{
"tokens" : [ {
  "token" : "sta",
  "start_offset" : 0,
  "end_offset" : 3,
  "type" : "word",
  "position" : 1
}, {
  "token" : "stac",
  "start_offset" : 0,
  "end_offset" : 4,
  "type" : "word",
  "position" : 2
}, {
  "token" : "stack",
  "start_offset" : 0,
  "end_offset" : 6,
  "type" : "word",
  "position" : 3
}, {
  "token" : "stacko",
  "start_offset" : 0,
  "end_offset" : 7,
  "type" : "word",
  "position" : 4
}, {
  "token" : "stackov",
  "start_offset" : 0,
  "end_offset" : 8,
  "type" : "word",
  "position" : 5
}, {