Is it possible to get the average of histogram_quantile of the last t minutes?

Question

What I'm intending to find is the average of the p99 latency in the last t minutes.

I tried this query, but it returned with the error "ranges only allowed for vector selectors"

avg_over_time(histogram_quantile(0.99, sum(rate(latency_buckets{service="foo"}[5m])) by (le))[5m])

From what I understand, what histogram_quantile does is return an instant value (let's say p99) and there is no way to get a series of p99 values over a specified interval. If so, are there any functions that can achieve the same goal?

Answer 1

It is possible using subquery syntax:

avg_over_time(instant_query[interval:resolution])

An example with your query (avg over 1h):

avg_over_time(
  histogram_quantile( # the instant query
    0.99,
    sum(
      rate(latency_buckets{service="foo"}[5m])
    ) by (le)
  )[1h:] # subquery [ interval : resolution (by default == scrape interval)]
)