How to monitor kube-apiserver in multi-master deployment?

Question

Saying for a K8s cluster with three master nodes, do I need to get metrics from each of the kube-apiserver running on all three nodes, or just one of them?

If only one of them, then do I need to worry about making sure always getting metric from the same node or let the dns decide which one to reach out to?

Is there a potential risk that the counter metrics on each nodes accumulate at a different rate and result in decrease in counters if pulling metrics from a random node each time?

Answer 1

You should monitor each of the API Server in all the three master nodes separately because if there is an issue in one of those you should probably remove it from the loadbalancer in-front of the API Servers.

You could use prometheus for monitoring. Kubernetes API Server exposes metrics in prometheus format already.