Scrolling or Pagination Elasticsearch Aggregations - Nest Framework

Question

My Elasticsearch document looks like this; I have created an average aggregation in the buckets of 60 seconds interval

{
"version" : 4,
"metric1" : 0.688872,
"metric2" : 0.021005,
"metric3" : 0.578913,
"metric4" : 71.57523,
"metric5" : 10.71166,
"Agentid" : "12345",
"epoch" : 1638173827064
}

My Aggregation Code looks like this;

var response = await _client.SearchAsync<MyPOCO>(s => s
                                        .Index(indexName)
                                        .TrackTotalHits(true)
                                        .Size(0)
                                        .Query(q => q
                                                .Bool(b => b
                                                    .Must(mu => mu
                                                        .Match(m => m
                                                            .Field("AgentId")
                                                            .Query(Convert.ToString(AgentId))
                                                        )
                                                    )
                                                    .Filter(fi => fi
                                                        .DateRange(r => r
                                                            .Field("epoch")
                                                            .GreaterThanOrEquals(Convert.ToString(StartTime))
                                                            .LessThan(Convert.ToString(EndTime))
                                                            .Format("epoch_millis")
                                                        )
                                                    )
                                                )
                                            )
                                        .Aggregations(a => a
                                        .DateHistogram("metricsperminute", ab => ab
                                             .Field("ts")
                                             .FixedInterval("1m")
                                             .Aggregations(x => x
                                                    .Average("metric1", m => m
                                                        .Field(o => o.metric1)
                                                    )
                                                    .Average("metric2", n => n
                                                        .Field(o => o.metric2)
                                                    )
                                                    .Average("metric3", o => o
                                                        .Field(o => o.metric4)
                                                    )
                                                    .Average("metric5", p => p
                                                        .Field(o => o.metric5)
                                                    )
                                            )))
                                        .Sort(sort => sort.Ascending("epoch")));

I am a bit lost in how to fetch more than 10k records/buckets in the aggregations? How to do pagination or scrolling? Please help.

Answer 1

There is a search.max_buckets dynamic cluster setting though I'd try to avoid returning too many buckets at once. It's extremely slow