[英]How to alert on metrics with a high cardinality in prometheus
when trying to create an alert on high metric cardinality with the expression count by(__name__) ({__name__=~".+"}) > 50
I get the error: vector contains metrics with the same labelset after applying rule labels
.当尝试使用表达式
count by(__name__) ({__name__=~".+"}) > 50
创建关于高度量基数的警报时,我收到错误: vector contains metrics with the same labelset after applying rule labels
。
As the expression works when using it directly in prometheus, I wonder if there is an actual way to use it in an alert?由于该表达式在 prometheus 中直接使用时有效,我想知道是否有实际的方法可以在警报中使用它?
I think I found a solution for this issue, as I was trying it myself.我想我找到了解决这个问题的方法,因为我自己也在尝试。
LT;DR LT; DR
use this promQL expression for alerting on metric cardinality:使用这个 promQL 表达式来警告度量基数:
label_replace(count by(__name__) ({__name__=~".+"}), "name", "$1", "__name__", "(.+)") > 50
Long Version长版
The issue as stated in the Prometheus error message. Prometheus 错误消息中所述的问题。 After the metric vector is converted to a vector of the alert, no labels differ and therefore are duplicated.
度量向量转换为警报向量后,没有标签不同,因此是重复的。 this means
这意味着
vector A ( metric_a{label=test}, metric_b{label=test} )
is converted in转换为
vector B ( alert_a{label=test}, alert_a{label=test})
and that is why you have duplicates这就是为什么你有重复
( caveat: that is at least my understanding) By adding a new label with the metric name itself, you create a unique label set. (警告:这至少是我的理解)通过添加带有度量名称本身的新标签,您可以创建一个唯一的标签集。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.