Prometheus monitoring Kubernetes Container Memory usage and report if container using more than 90%

Question

Looking for example how to monitor Container Memory Usage with Prometheus.

It reports all the containers ok if we using this query:

(container_memory_usage_bytes / container_spec_memory_limit_bytes) * 100 > 90

However works ok if container does not have a memory limit defined. The the divisor is 0, and the results are +Inf, meaning that the alert triggers incorrectly since +Inf matches > 90.

Any suggestions how to properly use Container Memory Usage monitoring?

Answer 1

I asked the same question from a different perspective just a few days earlier here . So far I have not found an answer. I have settled with adding another label has_memory_limit that I use to only alert on containers that have a limit defined.

---

Okay, I have figured it out:

((container_memory_usage_bytes / container_spec_memory_limit_bytes) != +Inf)  * 100 > 52

Since positive infinity, negative infinity and NaN are numbers in Prometheus , you can simply filter them out with != +Inf .