简体   繁体   English

如何使用过滤器查询优化弹性搜索aggs

[英]how to optimize elastic search aggs with filter query

When I tried to run this query elasticsearch does not answers, and a lot of cache eviction (related with field cache) occurs. 当我尝试运行此查询时,elasticsearch无法回答,并且发生大量缓存逐出(与字段缓存有关)。

I do not want to cache any fields in this query, because this is an analyze query which I will run only once a day. 我不想在此查询中缓存任何字段,因为这是一个分析查询,我每天只运行一次。 Is there any way to get this aggs without using field caching. 有什么办法可以在不使用字段缓存的情况下获取此摘要。 I tried doc values but it takes 45 second or more. 我尝试了doc值,但需要45秒或更长时间。 Look forward to your suggestions please. 请期待您的建议。

HEAP Memory : 30GB 堆内存:30GB
ES version 1.3.1 ES版本1.3.1
Size of index : 350 GB 索引大小:350 GB

{  
   "query": {
      "filtered": {
         "filter": {
            "bool": {
               "must": [
                  {
                     "term": {
                        "type": "161"
                     }
                  },
                  {
                     "term": {
                        "cat": "Math"
                     }
                  }
               ]
            }
         }
      }
   },
   "aggs": {
      "tepo": {
         "terms": {
            "field": "tepo"

         }
      }
   }
}            

You will need to use doc values or field data cache , without which aggregation cant happen. 您将需要使用doc值或字段数据缓存,否则将无法进行聚合。 As this query is not very frequent , it is advisable to disable filter cache for that query. 由于该查询不是很频繁,因此建议对该查询禁用过滤器缓存。

   "filter" : {
        "term" : {
            "cath" : "Math",
            "_cache" : false
        }
    }

This way filter bitset cache wont be generated and you can save some memory on it. 这样就不会生成过滤器位集缓存,您可以在上面保存一些内存。 LINK - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html#_caching_18 链接-http: //www.elasticsearch.org/guide/zh-CN/elasticsearch/reference/current/query-dsl-term-filter.html#_caching_18

Another approach would be to limit the amount of field data cache in main memory , and this way you can make sure field data cache wont get over board with the main memory LINK - http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html 另一种方法是限制主存储器中的现场数据缓存量,这样您就可以确保现场数据缓存不会被主存储器所占用。LINK- http://www.elasticsearch.org/guide/zh/ elasticsearch / guide / current / _limiting_memory_usage.html

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM