Elasticsearch: Filter array elements in nested documents

Question

Currently I'm trying to understand the structure of Elasticsearch documents and those responses. I created some indices and added a few documents to learn the basics. Now I am struggling with the responses of nested dicts.

Recently I added 2 documents to my existing index:

{
   {"id": 1,
    "vehicle": "car",
    "equipment": [
       {"v_id": 24, "gps": true,"color": "black"}
       {"v_id": 11, "gps": false, "color": "yellow"}
    ]
   },

  {"id": 2,
   "vehicle": "bus",
   "equipment": [
       {"v_id": 34, "gps": false,"color": "red"}
       {"v_id": 99, "gps": false,"color": "yellow"}
       {"v_id": 10, "gps": true,"color": "red"}
   ]
  }
}

Now I want to retrieve all yellow vehicles using the query:

{
   "query": {
     "match": {
         "equipment.color": "yellow“
      }
   }
}

I will get a response that looks like:

{
   {"id": 1,
    "vehicle": "car",
    "equipment": [
       {"v_id": 24, "gps": true,"color": "black"}
       {"v_id": 11, "gps": false, "color": "yellow"}
    ]
   },

  {"id": 2,
   "vehicle": "bus",
   "equipment": [
       {"v_id": 34, "gps": false,"color": "red"}
       {"v_id": 99, "gps": false,"color": "yellow"}
       {"v_id": 10, "gps": true,"color": "red"}
   ]
  }
}

because both documents contain a vehicle that has "yellow" as color value. Actually I try to filter some array-elements out from the response and want that the response looks like this:

{
   {"id": 1,
    "vehicle": "car",
    "equipment": [
       {"v_id": 11, "gps": false, "color": "yellow"}
    ]
   },

  {"id": 2,
   "vehicle": "bus",
   "equipment": [
       {"v_id": 99, "gps": false,"color": "yellow"}
   ]
  }
}

Is that possible with Elasticsearch or generally with dictionaries? If the answer is yes, how should the query-body look like?

Thanks a lot, cico

Answer 1

You need to use nested query along with inner_hits

Adding a working example with index mapping, search query, and search result

Index Mapping:

PUT myidx
{
  "mappings": {
    "properties": {
      "equipment":{
        "type":"nested"
      }
    }
  }
}

Search Query:

POST myidx/_search
{
  "query": {
    "nested": {
      "path": "equipment",
      "query": {
        "match": {
          "equipment.color": "yellow"
        }
      },
      "inner_hits": {}
    }
  }
}

Search Result:

{
  "took" : 432,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 0.87546873,
    "hits" : [
      {
        "_index" : "myidx",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 0.87546873,
        "_source" : {
          "id" : 1,
          "vehicle" : "car",
          "equipment" : [
            {
              "v_id" : 24,
              "gps" : true,
              "color" : "black"
            },
            {
              "v_id" : 11,
              "gps" : false,
              "color" : "yellow"
            }
          ]
        },
        "inner_hits" : {
          "equipment" : {
            "hits" : {
              "total" : {
                "value" : 1,
                "relation" : "eq"
              },
              "max_score" : 0.87546873,
              "hits" : [
                {
                  "_index" : "myidx",
                  "_type" : "_doc",
                  "_id" : "1",
                  "_nested" : {
                    "field" : "equipment",
                    "offset" : 1
                  },
                  "_score" : 0.87546873,
                  "_source" : {
                    "v_id" : 11,
                    "color" : "yellow",
                    "gps" : false                  // note this
                  }
                }
              ]
            }
          }
        }
      },
      {
        "_index" : "myidx",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 0.87546873,
        "_source" : {
          "id" : 2,
          "vehicle" : "bus",
          "equipment" : [
            {
              "v_id" : 34,
              "gps" : false,
              "color" : "red"
            },
            {
              "v_id" : 99,
              "gps" : false,
              "color" : "yellow"
            },
            {
              "v_id" : 10,
              "gps" : true,
              "color" : "red"
            }
          ]
        },
        "inner_hits" : {
          "equipment" : {
            "hits" : {
              "total" : {
                "value" : 1,
                "relation" : "eq"
              },
              "max_score" : 0.87546873,
              "hits" : [
                {
                  "_index" : "myidx",
                  "_type" : "_doc",
                  "_id" : "2",
                  "_nested" : {
                    "field" : "equipment",
                    "offset" : 1
                  },
                  "_score" : 0.87546873,
                  "_source" : {
                    "v_id" : 99,
                    "color" : "yellow",       // note this
                    "gps" : false
                  }
                }
              ]
            }
          }
        }
      }
    ]
  }
}