[英]Elasticsearch scroll api search “from”
我有一個腳本,該腳本基於URL索引http://example.com/sitemap.index.xml
生成站點地圖,其中index
是一個數字>0
,該數字定義每個塊應包含的結果。
$chunk = 10000;
$counter = 0;
$scroll = $es->search(array(
"index" => "index",
"type" => "type",
"scroll" => "1m",
"search_type" => "scan",
"size" => 10,
"from" => $chunk * ($index - 1)
));
$sid = $scroll['_scroll_id'];
while($counter < $chunk){
$docs = $es->scroll(array(
"scroll_id" => $sid,
"scroll" => "1m"
));
$sid = $docs['_scroll_id'];
$counter += count($docs['hits']['hits']);
}
// ...
現在,每次我訪問http://example.com/sitemap.1.xml
或http://example.com/sitemap.2.xml
,從ES返回的結果都是完全相同的。 它返回50
結果(每個分片10個),但似乎不接受from = 0
, from = 10000
計數。
我正在使用elasticsearch-php
作為ES庫。
有任何想法嗎?
在Java中,可以按以下步驟完成
QueryBuilder query = QueryBuilders.matchAllQuery();
SearchResponse scrollResp = Constants.client.prepareSearch(index)
.setTypes(type).setSearchType(SearchType.SCAN)
.setScroll(new TimeValue(600000)).setQuery(query)
.setSize(500).execute().actionGet();
while (true) {
scrollResp = Constants.client
.prepareSearchScroll(scrollResp.getScrollId())
.setScroll(new TimeValue(600000)).execute().actionGet();
System.out.println("Record count :"
+ scrollResp.getHits().getHits().length);
total = total + scrollResp.getHits().getHits().length;
System.out.println("Total record count: " + total);
for (SearchHit hit : scrollResp.getHits()) {
//handle the hit
}
// Break condition: No hits are returned
if (scrollResp.getHits().getHits().length == 0) {
System.out.println("All records are fetched");
break;
}
}
希望能幫助到你。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.