每个@timestamp小时的ElasticSearch查询聚合

Question

我正在对metricsbeat的elasticSearch进行查询，以便每小时对最常用的进程进行评级，在这一刻我将聚合每个进程的开始时间和进程名称，我需要使用字段“@timestamp”每小时“划分”这些组

这是我的实际查询

GET metricbeat*/_search?
{"query": {
          "bool": {
                "must": [
                    { "wildcard" : { "beat.hostname" : "ibmcx*" }},
                    { "range": {
                      "@timestamp": {
                        "gte": "2019-03-22T00:00:00",
                        "lte": "2019-03-23T00:00:00"}}},
                    {"terms" : { "beat.hostname" : ["ibmcxapp101", "ibmcxapp102", "ibmcxapp103",
                                    "ibmcxapp104", "ibmcxapp105", "ibmcxapp106", "ibmcxapp107",
                                    "ibmcxapp108", "ibmcxapp109", "ibmcxapp110", "ibmcxapp111",
                                    "ibmcxapp112", "ibmcxapp113", "ibmcxapp114", "ibmcxapp115",
                                    "ibmcxapp116", "ibmcxapp117", "ibmcxapp118", "ibmcxapp119",
                                    "ibmcxapp120", "ibmcxapp121", "ibmcxapp122", "ibmcxxaa100",
                                    "ibmcxxaa101", "ibmcxxaa102", "ibmcxxaa103", "ibmcxxaa104",
                                    "ibmcxxaa105", "ibmcxxaa106", "ibmcxxaa107", "ibmcxxaa108",
                                    "ibmcxxaa109", "ibmcxxaa110", "ibmcxxaa111", "ibmcxxaa112",
                                    "ibmcxxaa201", "ibmcxxaa202", "ibmcxxaa203", "ibmcxxaa204"
                                    ] }},
                    {"exists": {"field": "system.process.cmdline"}}
                ],
                "must_not": [
                   {"term" : { "system.process.username" : "NT AUTHORITY\\SYSTEM" }},
                   {"term" : { "system.process.username" : "NT AUTHORITY\\NETWORK SERVICE" }},
                   {"term" : { "system.process.username" : "NT AUTHORITY\\LOCAL SERVICE" }},
                   {"term" : { "system.process.username" : "NT AUTHORITY\\Servicio de red"}},
                   {"term" : { "system.process.username" : "" }}
                  ]
          }
        },
        "size": 0,
        "aggs": {
          "group_by_start_time": {
            "terms": {
              "field": "system.process.cpu.start_time"
            },
            "aggs": {
              "group_by_name": {
                "terms": {
                  "field": "system.process.name.keyword"
                }
              }
            }
          }
        },
        "size": 0,
        "sort" : [
            { "system.process.cpu.start_time" : {"order" : "asc"}},
            { "@timestamp" : {"order" : "asc"}},
            { "system.process.pid" : {"order" : "desc"}}
        ]}

Answer 1

它有点难以跟踪和重现 - 一个最小的例子（我认为整个query并不是真的需要），样本文档会有很长的路要走。

如果你想要每小时聚合，你需要做的第一件事是聚合，然后运行其他内部聚合。

每小时聚合的最小示例是：

POST /metricbeat*/_search?size=0
{
    "aggs" : {
        "metrics_per_hour" : {
            "date_histogram" : {
                "field" : "@timestamp",
                "interval" : "hour"
            }
        }
    }
}

在其他聚合中折叠将如下所示：

POST /metricbeat*/_search?size=0
{
    "aggs" : {
        "metrics_per_hour" : {
            "date_histogram" : {
                "field" : "@timestamp",
                "interval" : "hour"
            },
            "aggs" : {
                ...
            }
        }
    }
}

PS：如果您使用的是每日索引模式，则可以使用正确的日期而不是通配符，然后跳过查询的这一部分：

"range": {
    "@timestamp": {
        "gte": "2019-03-22T00:00:00",
        "lte": "2019-03-23T00:00:00"
    }
}