How To Get the Aggregated Usage of CPU in GRAFANA when hyperthreading is enabled

Question

We are running GRAFANA/PROMETHEUS to monitor our CPU metrics and find aggregated CPU Usage of all cpus. the problem is we have enabled hyperthreading and when we stress CPU the percentage exceeds from 100%. my question is how to limit that cpu usage to show only usage in 100% not more even if cpu is highly utilized.

PS i have tried setting the max and min limit in grafana but still the graph spikes goes above that limit.

Kindly give me the right query for this problem.

The queries I have tried are given below.

sum(irate(node_cpu_seconds_total{instance="localhost",job="node", mode!="idle"}[5m]))*100


100 - avg(irate(node_cpu_seconds_total{instance="localhost",job="node", mode!="idle"}[5m]))*100

and other similar queries we have tried.

Answer 1

If all you want is to "cap" a variable or expression result to a maximum value (that is, 100) you could simply use the Prometheus function clamp_max .

Thus, you could do:

clamp_max(<expr>, 100)

Answer 2

This is probably the most helpful query.

(1 - avg(irate(node_cpu_seconds_total{instance="$instance",job="$job",mode!="idle"}[5m])))*100

Replace your instance IP and your node exporter job name.