Prometheus 黑盒探测有用的指标

Question

I have around 1000 targets that are probed using HTTP.我有大约 1000 个使用 HTTP 探测的目标。

job="http_2xx", env="prod", instance="x.x.x.x"
job="http_2xx", env="test", instance="y.y.y.y"
job="http_2xx", env="dev", instance="z.z.z.z"

I want to know for the targets:我想知道目标：

Rate of failure by env in last 10 minutes.过去 10 分钟内 env 的失败率。
Increase in rate of failure by env in last 10 minutes. env 在过去 10 分钟内的失败率增加。
Curious what the following does:好奇以下是做什么的：

sum(increase(probe_success{job="http_2xx"}[10m]))

rate(probe_success{job="http_2xx", env="prod"}[5m]) * 100

The closest I have reached is with following to find operational by env in 10 minutes:我已经达到的最接近的是在 10 分钟内通过 env 找到操作：

avg(avg_over_time(probe_success{job="http_2xx", env="prod"}[10m]) * 100)

Answer 1

Rate of failure by env in last 10 minutes.过去 10 分钟内 env 的失败率。 The easiest way you can do it is:最简单的方法是：
sum(rate(probe_success{job="http_2xx"}[10m]) * 100) by (env)
This will return you the percentage off successful probes, which you can reverse adding *(-1) +100这将返回您成功探测的百分比，您可以反向添加*(-1) +100
Calculating rate over 10m and increase of rate over 10m seems redundant adding an increase function to the above query didn't work for me.计算超过 10m 的速率和超过 10m 的速率增加似乎是多余的，在上述查询中increase function 对我不起作用。 you can replace the rate function with increase if want to.如果需要，您可以将速率 function 替换为increase 。
The first query was pretty close it will calculate the increase of successful probes over 10m period.第一个查询非常接近，它将计算 10m 周期内成功探测的增加。 You can make it show increase of failed probes by adding == 0 and sum it by the "env" variable您可以通过添加== 0并通过“env”变量对其求和来使其显示失败探针的增加
sum(increase(probe_success{job="http_2xx"} == 0 [10m])) by (env)
Your second query will return percentage of successful request over 5m for prod environment您的第二个查询将为 prod 环境返回超过 5m 的成功请求百分比