弹性搜索边缘 ngram 未返回所有预期结果
[英]Elastic search edge ngram not returning all expected results
问题描述
I am having a hard time in finding the elastic search query unexpected results.
我很难找到弹性搜索查询意外结果。 Indexed the following documents into elastic search.将以下文档索引到弹性搜索中。
{
"group": "J00-I99", codes: [
{ "id": "J15", "description": "hello world" },
{ "id": "J15.0", "description": "test one world" },
{ "id": "J15.1", "description": "test two world J15.0" },
{ "id": "J15.2", "description": "test two three world J15" },
{ "id": "J15.3", "description": "hello world J18 " },
............................ // Similar records here
{ "id": "J15.9", "description": "hello world new" },
{ "id": "J16.0", "description": "new description" }
]
}
Here my aim is to implement autocomplete functionality and for that I used n-gram approach.我的目标是实现自动完成功能，为此我使用了 n-gram 方法。 I don't want to use complete suggester approach.我不想使用完整的建议方法。
Currently I am stuck with two issues:目前我遇到了两个问题：
- Search query (both id and description fields ) : J15搜索查询（id 和 description 字段）：J15
Expected result: All the above results which includes J15 Actual result: Getting only few results (J15.0, J15.1, J15.8)预期结果：以上所有结果，包括 J15 实际结果：得到的结果很少（J15.0、J15.1、J15.8）
- Search query (both id and description fields ) : test two搜索查询（id 和 description 字段）：测试两个
Expected result:预期结果：
{ "id": "J15.1", "description": "test two world J15.0" },
{ "id": "J15.2", "description": "test two three world J15" },
Actual Result:实际结果：
{ "id": "J15.0", "description": "test one world" },
{ "id": "J15.1", "description": "test two world J15.0" },
{ "id": "J15.2", "description": "test two three world J15" },
Then mapping is done like this.然后映射是这样完成的。
{
settings: {
number_of_shards: 1,
analysis: {
filter: {
ngram_filter: {
type: 'edge_ngram',
min_gram: 2,
max_gram: 20
}
},
analyzer: {
ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: [
'lowercase', 'ngram_filter'
]
}
}
}
},
mappings: {
properties: {
group: {
type: 'text'
},
codes: {
type: 'nested',
properties: {
id: {
type: 'text',
analyzer: 'ngram_analyzer',
search_analyzer: 'standard'
},
description: {
type: 'text',
analyzer: 'ngram_analyzer',
search_analyzer: 'standard'
}
}
}
}
}
}
Search Query:搜索查询：
GET myindex/_search
{
"_source": {
"excludes": [
"codes"
]
},
"query": {
"nested": {
"path": "codes",
"query": {
"bool": {
"should": [
{
"match": {
"codes.description": "J15"
}
},
{
"match": {
"codes.id": "J15"
}
}
]
}
},
"inner_hits": {}
}
}
}
Note: Document index will be large in size.注意：文档索引会很大。 Here only sample data mentioned.这里只提到了示例数据。
For the second issue, can i use multi_match with AND operator like the below?对于第二个问题，我可以像下面这样使用带有 AND 运算符的 multi_match 吗？
GET myindex/_search
{
"_source": {
"excludes": [
"codes"
]
},
"query": {
"nested": {
"path": "codes",
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "J15",
"fields": ["codes.id", "codes.description"],
"operator": and
}
}
]
}
},
"inner_hits": {}
}
}
}
Any help would be really appreciated as I am having hard time in fixing this.任何帮助将不胜感激，因为我很难解决这个问题。
3 个解决方案
解决方案1
1 2020-10-02 11:22:15
Issue was that by default inner_hits returns only 3 matching docs as mentioned in this official doc ,问题是默认情况下， inner_hits仅返回此官方文档中提到的 3 个匹配文档，
size
The maximum number of hits to return per inner_hits.
simply add size param in your inner_hits to get all the search results.只需在您的 inner_hits 中添加size参数即可获得所有搜索结果。
"inner_hits": {
"size": 10 // note this
}
Tried this on your sample data and see the search result for your first query which was returning only 3 search results在您的示例数据上尝试此操作并查看您的第一个查询的搜索结果，该查询仅返回 3 个搜索结果
First query search result第一次查询搜索结果
"hits": [
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 2
},
"_score": 1.8687118,
"_source": {
"id": "J15.1",
"description": "test two world J15.0"
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 3
},
"_score": 1.7934312,
"_source": {
"id": "J15.2",
"description": "test two three world J15"
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 0
},
"_score": 0.29618382,
"_source": {
"id": "J15",
"description": "hello world"
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 1
},
"_score": 0.29618382,
"_source": {
"id": "J15.0",
"description": "test one world"
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 4
},
"_score": 0.29618382,
"_source": {
"id": "J15.3",
"description": "hello world J18 "
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 5
},
"_score": 0.29618382,
"_source": {
"id": "J15.9",
"description": "hello world new"
}
}
]
}
}
}
}
解决方案2
1 已采纳 2020-10-02 12:43:56
Adding another answer, as its a different issue and first answer was focused on first issue.添加另一个答案，因为它是一个不同的问题，第一个答案集中在第一个问题上。
Issue is that your second query test two returns test one world as well as while indexing you are using the ngram_analyzer which is using the standard analyzer which split the text on white-spaces and again your search analyzer is standard so if you use the Analyze API on your indexed doc and search term, you will see it matches the tokens:问题是您的第二个查询test two返回test one world以及在索引时您使用的是ngram_analyzer ，它使用标准分析器将文本拆分为空格，并且您的搜索分析器再次是standard因此如果您使用分析 API在您的索引文档和搜索词中，您将看到它与标记匹配：
{
"text" : "test one world",
"analyzer" : "standard"
}
And generated tokens和生成的令牌
{
"tokens": [
{
"token": "test",
"start_offset": 0,
"end_offset": 4,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "one",
"start_offset": 5,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "world",
"start_offset": 9,
"end_offset": 14,
"type": "<ALPHANUM>",
"position": 2
}
]
}
And for your search term test two并为您的搜索词test two
{
"tokens": [
{
"token": "test",
"start_offset": 0,
"end_offset": 4,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "two",
"start_offset": 5,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 1
}
]
}
As you can see test token was present in your document hence you get that search result.正如您所看到的，您的文档中存在test令牌，因此您会得到该搜索结果。 and it can be solved by using the AND operator in the query as shown below它可以通过在查询中使用 AND 运算符来解决，如下所示
Search query搜索查询
{
"_source": {
"excludes": [
"codes"
]
},
"query": {
"nested": {
"path": "codes",
"query": {
"bool": {
"must": {
"multi_match": {
"query": "test two",
"fields": [
"codes.id",
"codes.description"
],
"operator" :"AND"
}
}
}
},
"inner_hits": {}
}
}
}
And search results和搜索结果
"hits": [
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 2
},
"_score": 2.6901608,
"_source": {
"id": "J15.1",
"description": "test two world J15.0"
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 3
},
"_score": 2.561376,
"_source": {
"id": "J15.2",
"description": "test two three world J15"
}
}
]
}
}
}
}
解决方案3
0 2020-10-02 11:20:26
Adding a working example with index mapping, search query, and search result添加带有索引映射、搜索查询和搜索结果的工作示例
Index Mapping:索引映射：
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "my_tokenizer"
}
},
"tokenizer": {
"my_tokenizer": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 20,
"token_chars": [
"letter",
"digit"
]
}
}
},
"max_ngram_diff": 50
},
"mappings": {
"properties": {
"group": {
"type": "text"
},
"codes": {
"type": "nested",
"properties": {
"id": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
}
}
Index Data:指数数据：
{
"group": "J00-I99",
"codes": [
{
"id": "J15",
"description": "hello world"
},
{
"id": "J15.0",
"description": "test one world"
},
{
"id": "J15.1",
"description": "test two world J15.0"
},
{
"id": "J15.2",
"description": "test two three world J15"
},
{
"id": "J15.3",
"description": "hello world J18 "
},
{
"id": "J15.9",
"description": "hello world new"
},
{
"id": "J16.0",
"description": "new description"
}
]
}
Search Query:搜索查询：
{
"_source": {
"excludes": [
"codes"
]
},
"query": {
"nested": {
"path": "codes",
"query": {
"bool": {
"should": [
{
"match": {
"codes.description": "J15"
}
},
{
"match": {
"codes.id": "J15"
}
}
],
"must": {
"multi_match": {
"query": "test two",
"fields": [
"codes.id",
"codes.description"
],
"type": "phrase"
}
}
}
},
"inner_hits": {}
}
}
}
Search Result:搜索结果：
"inner_hits": {
"codes": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 3.2227304,
"hits": [
{
"_index": "stof_64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 3
},
"_score": 3.2227304,
"_source": {
"id": "J15.2",
"description": "test two three world J15"
}
},
{
"_index": "stof_64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 2
},
"_score": 2.0622847,
"_source": {
"id": "J15.1",
"description": "test two world J15.0"
}
}
]
}
}
}
}
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.