如何在Elasticsearch中进行多级子聚合？

Question

I have a USER type and it has children of EVENT type.

The events can have different fields. So one event might be have a visitDate field, and another event might be a demographics event and have an age field.

I want to do multi-level aggregations, where I can see, for example, the breakdown of ages each day. Ie:

1st July
  Age: 24 - 10 docs
  Age: 25 - 15 docs 
2nd July
  Age: 24 - 5 docs
  Age: 25 - 6 docs
etc

The problem I am facing is that if I do a children aggregation like:

{
  "size": 2,
  "aggs": {
    "events": {
      "children": {
        "type" : "event"
      },
      "aggs": {
        "visitDate": {
          "terms":{
            "field":"visitDate",
            "size":0
          },
          "aggs":{
            "byAge":{
              "terms":{
                "field":"age",
                "size":0
              } 
            } 
          } 
        }
      }
    }
  }
}

etc

the second-level aggregation (on age) only has access to the child documents that matched the aggregation on visitDate, NOT any of the other child documents that are children of the same parents.

How can I aggregate arbitrarily deeply, but with each successive aggregation looking at the PARENTS of the children in that bucket?

Answer 1

You can achieve that using the nested aggregation, I had the same case before and I used the following query to get such result (aggregating audio book time usage by language which is a nested child doc in the audio_book):

Elasticsearch doc:

  {
    "_index": "dev_analytics",
    "_type": "UsageAnalytics",
    "_id": "AWLc_3OL-IIrUiI24XbD",
    "_score": 2.1704133,
    "_source": {
      "timestamp": 1523983693559,
      "audio_book": {
        "id": "0032423404234234234",
        "name": "Jag ger dig solen",
        "languages": [
          {
            "code": "sv",
            "type": "TEXT"
          }
        ],
      "usage": {
        "unitOfMeasure": "SEC",
        "totalUsage": 1200
      }
    }
  }

Required aggregation:

"buckets": [
          {
            "key": "sv",
            "doc_count": 247,
            "usage": {
              "doc_count": 247,
              "total_usage": {
                "count": 247,
                "min": 1200,
                "max": 1200,
                "avg": 1200,
                "sum": 296400
              }
            }
          },
          {
            "key": "en",
            "doc_count": 47,
            "usage": {
              "doc_count": 47,
              "total_usage": {
                "count": 47,
                "min": 1200,
                "max": 1200,
                "avg": 1200,
                "sum": 56400
              }
            }
          },
          {
            "key": "de",
            "doc_count": 24,
            "usage": {
              "doc_count": 24,
              "total_usage": {
                "count": 24,
                "min": 1200,
                "max": 1200,
                "avg": 1200,
                "sum": 28800
              }
            }
          },
          {
            "key": "ar",
            "doc_count": 20,
            "usage": {
              "doc_count": 20,
              "total_usage": {
                "count": 20,
                "min": 1200,
                "max": 1200,
                "avg": 1200,
                "sum": 24000
              }
            }
          }
        ]

Nested aggregation query:

GET dev_analytics/UsageAnalytics/_search
{
  "size": 0,
  "aggs": {
     "lang": {
       "nested": {
         "path": "title.languages"
       },
       "aggs": {
         "terms": {
           "terms": {
             "field": "title.languages.code"
           },
           "aggs": {
             "usage": {
               "reverse_nested": {},
               "aggs": {
                 "total_usage": {
                   "stats": {
                     "field": "usage.totalUsage"
                   }
                 }
               }
             }
           }
         }
       }
     }
   }
}