Elasticsearch分页自-大小结果窗口太大

Question

I would like to have pages for my documents, each Page should have 1000 results.

Documents size: 95.000 docs

So starting from product number 10k, I would like to have 1K results from that point, however, using this query, I got the error mentioned below

Query:

this.es.search({
  index: indexName,
  type: type,
  from: 10000,
  size: 1000,
  body: {}
});

Error:

{
    "msg": "[query_phase_execution_exception] Result window is too large, from + size must be less than or equal to: [10000] but was [16000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.",
    "path": "/products/Product/_search",
    "query": {},
    "body": "{\"from\":10000,\"size\":1000}",
    "statusCode": 500,
    "response": "{\"error\":{\"root_cause\":[{\"type\":\"query_phase_execution_exception\",\"reason\":\"Result window is too large, from + size must be less than or equal to: [10000] but was [16000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"products\",\"node\":\"bTGkra6dTB-0Z_8jVUNByQ\",\"reason\":{\"type\":\"query_phase_execution_exception\",\"reason\":\"Result window is too large, from + size must be less than or equal to: [10000] but was [16000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.\"}}]},\"status\":500}"
}

How can I increase the paging result/ limit?

Also any other method is welcome, thank you.

Answer 1

Based on elasticsearch documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html

Note that from + size can not be more than the index.max_result_window index setting which defaults to 10,000

As alternative, you can try to use scroll or searchAfter . For more real-time query, searchAfter is more suitable.

Scroll documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

SearchAfter documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html

Answer 2

As the error shows Elastic will only display 10000 rows.

so to display more than that number you need to use the scroll function

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

An example of a js search with scroll can be found here.

async await with elasticsearch search/scroll

Answer 3

好吧，我可以使用此CURL增加分页的限制

curl -XPUT "http://localhost:9200/my_index/_settings" -d '{ "index" : { "max_result_window" : 500000 } }'

Answer 4

Increase max_result_window as below, it works

PUT indexName/_settings
{
  "index": {
    "max_result_window": 10000000
  }
}