Analyse kubernetes pod network traffic

Question

All,

Off late my cloud provider is charging more for data transfer. Finally noticed one of the K8 pod is having more data transfer. Is there a way i can find out pod level network traffic such us how much transmitted and received bytes with native kubernetes command?

Thanks Bala

Answer 1

kubectl top command shows usage , not allocation. Allocation is what causes the insufficient CPU problem. There's a ton of confusion in this issue about the difference.

AFAICT, there's no easy way to get a report of node CPU allocation by pod, since requests are per container in the spec. And even then, it's difficult since .spec.containers[*].requests may or may not have the limits / requests fields. But there is something like kubectl-view-allocations with it you can explore your kube resource usage and allocation. It can provide result grouped by namespaces, nodes, pods and filtered by resources'name.

kubectl-view-allocations [FLAGS] [OPTIONS]

FLAGS:
    -h, --help         Prints help information
    -z, --show-zero    Show lines with zero requested and zero limit and zero allocatable
    -V, --version      Prints version information

OPTIONS:
    -g, --group-by <group-by>...              Group informations (hierarchicaly) (default: -g resource -g node -g pod)
                                              [possible values: resource, node, pod]
    -n, --namespace <namespace>               Show only pods from this namespace
    -r, --resource-name <resource-name>...    Filter resources shown by name(s), by default all resources are listed

In your case I think safest option is to install Heapster or metrics-server , cAdvisor and Grafana .

Heapster enables Container Cluster Monitoring and Performance Analysis for Kubernetes (versions v1.0.6 and higher), and platforms which include it.

Heapster collects and interprets various signals like compute resource usage - with this metrics you will find the problem wit htransfering more data by specific pods, lifecycle events, etc. Heapster supports multiple sources of data.

Container metrics are available mostly through cAdvisor. cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers. Specifically, for each container it keeps resource isolation parameters, historical resource usage, histograms of complete historical resource usage and network statistics. This data is exported by container and machine-wide.

Grafana on the other hand allows you to query, visualize, alert on and understand gathered metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data driven culture.

Take a look: kubernetes-metrics , metrics-server-installation .