在Elasticsearch中查询日期范围

Question

I want to get documents from last 30 days in elastic search but it returns empty.

it is my mapping:

PUT /books
{
    "mappings": {
        "impressions": {
            "properties": {

                "booksCreated" : {
                  "type": "date",
                  "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis",
                  "index": true

                }
            }
        }
    }
}

and it is my query:

POST /books/_search?size=0
{
    "aggs": {
        "range": {
            "date_range": {
                "field": "booksCreated",
                "format": "yyyy-MM-dd",
                "ranges": [
                    { "to": "now" }, 
                    { "from": "now-1M/M" } 
                ]
            }
        }
    }
}

I've tried all possible ways but it returns empty.

but i can query on @timestamp field

the problem is that logstash changes the field type from date to string. my json is :

{
    "index":"books",
    "type":"book",
    "body":{
    "impressions":{
    "_source":{
    "enabled":true
    },
    "properties":{
    "BookCreated":"2017-09-18 12:18:39"
    }
    }
  }
 }

and my logstash config:

input {
    file {
        path => "E:\data2\log\logstash.log"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        codec => json
    }
}

filter {
    mutate {
         strip => ["message"]
    }
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "books"
        document_type => "book"         
     }

}

i will log the json to a log file and logstash send them to elasticsearch

after adding json the mapping chasnges to this:

{
  "Books": {
    "mappings": {
      "Books": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "BookCreated": {
            "type": "date",
            "format": "yyyy-MM-dd HH:mm:ss"
          },
          "body": {
            "properties": {
              "Books": {
                "properties": {
                  "_source": {
                    "properties": {
                      "enabled": {
                        "type": "boolean"
                      }
                    }
                  },
                  "properties": {
                    "properties": {
                      "BookCreated": {
                        "type": "text",
                        "fields": {
                          "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          },
          "host": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "index": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "path": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "type": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}

it has two BookCreated one isdate and the other is text

Answer 1

You need to put from and to in the same range, like this:

POST /books/_search?size=0
{
    "aggs": {
        "range": {
            "date_range": {
                "field": "BookCreated",
                "format": "yyyy-MM-dd",
                "ranges": [
                    { 
                      "from": "now-1M/M",
                      "to": "now"
                    } 
                ]
            }
        }
    }
}

Answer 2

I'm pretty sure there is an issue with your mapping. First of all, make sure the bookCreated field is named consistently, both in regards to naming as well as capitalization!

Secondly, I believe the reason you have two bookCreated is because your mapping contains a bookCreated property. Your JSON however contains a nested structure: body => properties => bookCreated . Either flatten/transform the book in logstash to the required index structure, or model your index according to your json, which could look something like this?

"mappings": {
  "properties": {
    "body": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "bookCreated": {
              "type": "date",
              "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis",
              "index": true
            }
          }
        }
      }
    }
  }
}

Either way, I recommend you to set "dynamic": "strict" so you will actually see when you make a mistake in the mapping rather than just new fields being created