如何在带有和不带有白色空格的python中的Elasticsearch中应用搜索

Question

I am building a search system using ElasticSearch in python. I loaded a csv and created an index for search.

from elasticsearch import helpers, Elasticsearch
import csv

es = Elasticsearch()

with open('/Users/anubhav/Office/elasticsearch-5.6.0/all_products.csv') as f:
    reader = csv.DictReader(f)
    helpers.bulk(es, reader, index='product-index', doc_type='product-index')

es.indices.create(
    index='product-index',
    body={
      "settings": {
        "analysis": {
          "analyzer": {
            "my_analyzer": {
              "tokenizer": "my_tokenizer"
            }
          },
          "tokenizer": {
            "my_tokenizer": {
              "type": "edge_ngram",
              "min_gram": 2,
              "max_gram": 10,
              "token_chars": [
                "letter",
                "digit",
                "whitespace"
              ]
            }
          }
        }
      }
    },
    # Will ignore 400 errors, remove to ensure you're prompted
    ignore=400
)

response = es.search(
index='product-index',
body={
    "query": {
        "match": {
            "product": "PD5MP2 price"
        }
    },
    "aggs": {
        "top_10_states": {
            "terms": {
                "field": "state",
                "size": 10
            }
        }
    }
}
)

print response

The csv looks something like:

when I do search using:

res = es.search(index="product-index", doc_type="product-index", body={"query": {"match": {"product": "DD 350"}}})

this works fine because the exact product is there in CSV. But when I change the query to

res = es.search(index="product-index", doc_type="product-index", body={"query": {"match": {"product": "DD350"}}})

It doesn't works. Can someone please help me with this?

Answer 1

By default Elasticsearch uses Standant Tokenizer which divides text into terms on word boundaries, as defined by the Unicode Text Segmentation algorithm. That means DD 350 will be tokenized as DD, 350 . When you search, search keywords also will be tokenized as same tokenizer. So when you search for DD 350 Elasticsearch will look for both DD and 350 and both is in your index. But when you search for DD350 Elasticsearch won't be able to find it index. You should check the tokenizers and their usage.