对具有某些日期范围的索引进行弹性搜索查询

Question

I have below scenario:

I have following indices in elastic search.

• index-2016.04.10
• index-2016.04.11
• index-2016.04.12
• index-2016.04.15
• index-2016.04.16
• index-2016.04.18

Now suppose , I want to search some data in elastic search between dates - 2016.04.11 to 2016.04.16. My questions are:

1. Do we have any way to run a single query and define some filter parameters so that the search will happen only in the indices between these two dates ?

2. If not, then how can we optimize the search query If we need to search the data in some range of indices ?

3. Java implementation.

Please help..

Answer 1

I see two options.

You specify indexes when you search such as

GET /index-2016.04.10,index-2016.04.11,index-2016.04.12/_search?ignore_unavailable=true
{
    "query": {
        yourquery
    }
}

Or Filter in query (But this approach could be slow, and depends on amount of indexes could throw shard exception, since you will query all indexes which match pattern)

GET /index-*/_search
{
    "query": {
        "terms" : {
            "_index" : ["index1", "index2"]
        }
    }
}

I assume you will be indexing data in the past like last 7, 14, 30 days: so in this case i would definitely go with first approach, do index name calculation in your app

UPDATE 1: To prevent error for non existing indicies you can set flag ignore_unavailable

UPDATE 2: Well i you need to do search in the past one of the solution could be haveing an agregation job.

In ES there is reindex api

POST _reindex
{
  "source": {
    "index": ["twitter", "blog"]
  },
  "dest": {
    "index": "all_together"
  }
}

You will have daily index up to 7 days. Then on Monday 0:0 you aggregate data to weekly index.

You will have weekly up to 5 indexes. Again last day of the month you reindex to monthly index.

In query you combine multiple approach by providing which indexes you want to search and query filter.

Answer 2

You can use aliasses and then query to alias like an index.

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-add-alias.html#indices-add-alias