Get total memory usage per node in prometheus

Question

I want to query the total memory usage per node in the last week.

Here's the query I am using:

sum(container_memory_working_set_bytes{container_name!="POD"}) by (node)

Interestingly, the numbers returned are larger than the physical memory of the nodes.

I am using m5.large EC2 instances(2core, 8GB ram) for all my kubernetes nodes.

What would be the correct query to get total memory per node?

Answer 1

You are already using the best approximation for example according to this blogpost . The issue is that even depending on the software the respective container is running (Python, Golang, etc.) the way to measure the "real" memory usage differs.

But from what I understand container_memory_working_set_bytes is the best choice for getting data about individual containers.