[英]Extract keywords (multi word) from text using elastic search
I have an index full of keywords and based on those keywords I want to extract the keywords from the input text. 我有一个充满关键字的索引,并根据这些关键字我想从输入文本中提取关键字。
Following is the sample keyword index. 以下是示例关键字索引。 Please note that the keywords can be of multiple words too, or basically they are tags which are unique. 请注意,关键字也可以是多个单词,或者基本上它们是唯一的标签。
{
"hits": {
"total": 2000,
"hits": [
{
"id": 1,
"keyword": "thousand eyes"
},
{
"id": 2,
"keyword": "facebook"
},
{
"id": 3,
"keyword": "superdoc"
},
{
"id": 4,
"keyword": "quora"
},
{
"id": 5,
"keyword": "your story"
},
{
"id": 6,
"keyword": "Surgery"
},
{
"id": 7,
"keyword": "lending club"
},
{
"id": 8,
"keyword": "ad roll"
},
{
"id": 9,
"keyword": "the honest company"
},
{
"id": 10,
"keyword": "Draft kings"
}
]
}
}
Now, if I input the text as "I saw the news of lending club on facebook, your story and quora" the output of the search should be ["lending club", "facebook", "your story", "quora"] . 现在,如果我输入文本为“我在facebook上看到了借阅俱乐部的新闻,你的故事和quora” ,搜索的输出应该是[“借阅俱乐部”,“脸书”,“你的故事”,“quora”] 。 Also the search should be case insensetive 此外,搜索应该是案例性的
There's just one real way to do this. 只有一种方法可以做到这一点。 You'll have to index your your data as keywords and search it analyzed with shingles: 您必须将数据编入索引作为关键字并使用带状疱疹进行搜索:
See this reproduction: 看到这个复制品:
First, we'll create two custom analyzers: keyword and shingles: 首先,我们将创建两个自定义分析器:关键字和带状疱疹:
PUT test
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer_keyword": {
"type": "custom",
"tokenizer": "keyword",
"filter": [
"asciifolding",
"lowercase"
]
},
"my_analyzer_shingle": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"asciifolding",
"lowercase",
"shingle"
]
}
}
}
},
"mappings": {
"your_type": {
"properties": {
"keyword": {
"type": "string",
"index_analyzer": "my_analyzer_keyword",
"search_analyzer": "my_analyzer_shingle"
}
}
}
}
}
Now let's create some sample data using what you gave us: 现在让我们使用您提供的内容创建一些示例数据:
POST /test/your_type/1
{
"id": 1,
"keyword": "thousand eyes"
}
POST /test/your_type/2
{
"id": 2,
"keyword": "facebook"
}
POST /test/your_type/3
{
"id": 3,
"keyword": "superdoc"
}
POST /test/your_type/4
{
"id": 4,
"keyword": "quora"
}
POST /test/your_type/5
{
"id": 5,
"keyword": "your story"
}
POST /test/your_type/6
{
"id": 6,
"keyword": "Surgery"
}
POST /test/your_type/7
{
"id": 7,
"keyword": "lending club"
}
POST /test/your_type/8
{
"id": 8,
"keyword": "ad roll"
}
POST /test/your_type/9
{
"id": 9,
"keyword": "the honest company"
}
POST /test/your_type/10
{
"id": 10,
"keyword": "Draft kings"
}
And finally query to run search: 最后查询运行搜索:
POST /test/your_type/_search
{
"query": {
"match": {
"keyword": "I saw the news of lending club on facebook, your story and quora"
}
}
}
And this is result: 这是结果:
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 0.009332742,
"hits": [
{
"_index": "test",
"_type": "your_type",
"_id": "2",
"_score": 0.009332742,
"_source": {
"id": 2,
"keyword": "facebook"
}
},
{
"_index": "test",
"_type": "your_type",
"_id": "7",
"_score": 0.009332742,
"_source": {
"id": 7,
"keyword": "lending club"
}
},
{
"_index": "test",
"_type": "your_type",
"_id": "4",
"_score": 0.009207102,
"_source": {
"id": 4,
"keyword": "quora"
}
},
{
"_index": "test",
"_type": "your_type",
"_id": "5",
"_score": 0.0014755741,
"_source": {
"id": 5,
"keyword": "your story"
}
}
]
}
}
So what it does behind the scenes? 幕后它做了什么?
é
becomes e
) and lowercase filter (case insensitive search). 我还添加了asciifolding过滤器,因此它标准化字母,即é
变为e
)和小写过滤器(不区分大小写的搜索)。 So for instance Draft kings
is indexed as draft kings
因此,例如Draft kings
被列为draft kings
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.