How to calculate and show the value of counter as it changed in grafana chart

Question

I have a counter metric upgrade_counter which is incremented every time an upgrade happens. I want to show up a chart in grafana which will tell how many upgrades we have got in last 7 days. I want to show a progression as how the num of upgrades increased over the last 7 days.

I have used following query and it does not correctly show how many upgrades we have received.

sum(increase(upgrade_counter{}[7d]))

Value reported in chart is around 40 - 50% lower.

Answer 1

Prometheus may return unexpected results from increase() function because of extrapolation - see this issue . I'd suggest trying VictoriaMetrics instead - it returns the expected results from increase() function.

Note that the PromQL query is executed independently per each point on the graph in Grafana. So every point on the graph for increase(upgrade_counter[7d]) query would contain the number of upgrades during the last 7 days ending at that particular point. If you need obtaining a cumulative graph, which starts from zero and increases over time with the number of upgrades on the selected time range, then try the following MetricsQL query:

running_sum(increase(upgrade_counter))

This query works only in VictoriaMetrics, and it doesn't work in Prometheus, because it uses the following MetricsQL features:

• running_sum() function
• implicit lookbehind window in square brackets inside increase() function. See these docs for details.