Issues with null value in Elasticsearch

Question

Here's an example of my data :

{
    "MOD_DATE_START": "2010-04-20T15:05:49Z",
    "MOD_DATE_END": null,
    "MOD_ID": "123456789",
}

I'm having some issues with my Elasticsearch query. I have a couple of date fields where I am doing a range based filtering to make sure that my date is in between the start and end dates.

My first query (which works well) is filtering on the :

curl -s -XPOST http://server:9200/myindex/mytype/_search?pretty=true -d '
{
  "fields": ["MOD_ID", "MOD_DATE_START", "MOD_DATE_END"], 
    "query": {
        "bool": {
            "must": [
                {"term": {"MOD_ID": "123456789"}},
                {"range": {"MOD_DATE_START": {"lte": "2012-04-20T15:05:49Z"}}}
            ]
        }
    }
}
'

The MOD_DATE_START field always contains information, so the first query works well.

Since the second date field, MOD_DATE_END, is null in most cases I would like to modify my query too add the following test :

IF "MOD_DATE_END" NOT NULL then
    {"range": {"MOD_DATE_END": {"gte": "2012-04-20T15:05:49Z"}}}
ELSE skip "MOD_DATE_END"

I am, however, not quite able to figure out how to modify my query to add the third condition to be able to perform the gte test successfully.

Thanks in advance for your help.

Answer 1

One way to achieve this is by using a missing filter in a filtered query . Example below :

 curl -s -XPOST http://server:9200/myindex/mytype/_search?pretty=true -d '
    {
      "fields": ["MOD_ID", "MOD_DATE_START", "MOD_DATE_END"], 
       "query": {
          "filtered": {
             "filter": {
                "bool": {
                   "must": {
                      "range": {
                         "MOD_DATE_START": {
                            "lte": "2012-04-20T15:05:49Z"
                         }
                      }
                   },
                   "should": [
                      {
                         "missing": {
                            "field": "MOD_DATE_END",
                            "null_value": true,
                            "existence": true
                         }
                      },
                      {
                         "range": {
                            "MOD_DATE_START": {
                               "gte": "2012-04-20T15:05:49Z"
                            }
                         }
                      }
                   ]
                }
             },
             "query": {
                "term": {
                   "MOD_ID": "123456789"
                }
             }
          }
       }
    }

'