I'm trying to retrieve all data from the elasticsearch based on a message occurrence, i figured that if i used Scroll i could loop until the document search end but the following query returns Documents = 0 but Total = 1954:
var response = client.Search<Log4Net>(s => s
.Query(q => q.QueryString(qs => qs
.DefaultField(m => m.Message).Query("\"" + message + "\"")))
.SearchType(SearchType.Scan)
.Scroll("60s"));
while (response.Documents.Any())
{
var request = new BulkRequest();
request.Refresh = true;
request.Consistency = Consistency.One;
request.Operations = new List<IBulkOperation>();
foreach (var item in response.Documents)
{
request.Operations.Add(new BulkIndexOperation<Log4Net>(item));
}
var result = client.Bulk(request);
response = client.Scroll<Log4Net>("60s", response.ScrollId);
}
the response.Document is coming empty if i use the scroll, if i remove and get the first 1000 messages i can get the data, is anything wrong with how i'm using the Scroll?
If you specify .SearchType(SearchType.Scan)
, the first response doesn't contain any documents; It will give you the total documents in the .Total
property that will be returned by scrolling using the .ScrollId
on the response in a scroll request.
If you don't specify .SearchType(SearchType.Scan)
, the first response will contain the first set of documents.
This is a difference in Elasticsearch and not NEST. SearchType.Scan
is actually deprecated in 2.1.0 , but is still in NEST 2.x as it supports all minor versions of Elasticsearch 2.x.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.