Elasticsearch post_filter on nested types not filtering on aggregations

Question

Hello Elasticsearch gurus, I need your help! We're using Elasticsearch v5.4 and I have multiple nested types in elasticsearch in the form of:

"testLookup" : [
  {
    "id": 1001,
    "name": "test1"
  },
  {
    "id": 1002,
    "name": "test2"
  }
]

which I'm trying to display the names as checkbox options in a filter.

I was trying to use the post_filter top level element to basically filter the aggregations so I can update and display ONLY the affected filter options, kinda like how it is being used on this demo site: http://demo.searchkit.co

  "post_filter": {
    "bool":{
      "must": [
     {    
      "nested" : {
        "path" : "testLookup",
        "query": {
          "bool": {
            "filter":{ 
              "bool":{
                "should":[
                        { "term":  { "perTestLookup.name.keyword": "Adele"}}
         ]
            }
        }
      }
    }
    }
     },
              {    
    "nested" : {
      "path" : "testLookup2",
      "query": {
        "bool": {
          "filter":{ 
            "bool":{
              "should":[         
                { "term":  { "perTestLookup2.name.keyword": "Gene" }}
         ]
            }
        }
      }
    }
    }     
    }
  ]
 }
}

If i'm not mistaken I think what the post_filter does is apply the filter for each of the aggregations before the search request is sent, which you can observe that by looking into the request payload of the search filter after you click one of the filter checkboxes.

However, I can not get the post_filter to apply the filters to the aggregations, the filters are just not being applied. Why is this the case? is post_filter for nested types not supported?

Any tips or guidance will be greatly appreciated. Thank you!

Answer 1

Note: I do not have any searchkit experience.

The post_filter in an Elasticsearch request is executed after the aggregations are calculated to reduce the hits result further. See the docs about post_filter .

The idea behind this is the fact that in a faceted search (ie imagine filtering by category), you still want to retrieve that category counts for all categories in your aggregations, but you want to filter for that clicked category in your the documents you are searching.

So this is expected behaviour from the Elasticsearch perspective.