如何在Grafana中创建SLA图表？

Question

I have a seriesMetric , and a constantLine (the goal/SLA). I want to be able to "count" the occurances below the line, and divide by the total to get a % within SLA. I know how to do the 2nd part, using asPercent , but I can't figure out how to count the occurrences below the line in Grafana. There's no countif function. I tried currentBelow and averageBelow , but then nothing appeared on my chart (the two lines disappeared). I assume I did something wrong.

I think this is a pretty common use case, and I assume someone has done this. Ideally the % would go in a singleStat panel next to the chart.

(We use Gafana with Graphite.)

Sample SLA Chart

Answer 1

We finally figured this out, thanks to a coworker. Here's what worked for us:

"A", "target": "averageSeries(stats.timers.rails.production-aws.rack.*.util.mean)"

This is the main metric.

"B", "target": "removeAboveValue(#A,25)"

This removes the values that are out of SLA. Our SLA is utilization should be <25%.

"C", "target": "alias(transformNull(asPercent(#B), 0), '% within SLA')"

asPercent divides the series by itself which creates 1's (and Nulls).

transformNull operates off the removeAboveValue function replacing Nulls with 0s.

Finally, under Axis & Grid > Legend > Click Average and Show 2 Decimals. Doing this will provide an average SLA attainment for whatever timeframe you're viewing.

Here is the result. SLA Chart

Answer 2

Depending on your what exactly your SLA will measure, this can be solved in a any number of ways. A general answer to this question, and also the way I've solved it in projects, is to have an external script pull some relevant metrics from Graphite and simply push the result back (with a new key like xxsla.satisfied) as a pure binary result; either 0 or 1. Then it's a simple matter of just averaging the value of the series and displaying that in percentage form. Because in the end, you either meet the SLA or you don't. If you want to dig deeper, you can always look at the graphs like the one you had in the picture.