[英]How does Kubernetes Horizontal Pod Autoscaler calculate CPU Utilization for Multi Container Pods?
Question 1.)问题 1.)
Given the scenario a multi-container pod, where all containers have a defined CPU request:给定一个多容器 pod 的场景,其中所有容器都有一个定义的 CPU 请求:
How would Kubernetes Horizontal Pod Autoscaler calculate CPU Utilization for Multi Container pods? Kubernetes Horizontal Pod Autoscaler 如何计算多容器 Pod 的 CPU 利用率?
Does it average them?它是平均的吗? (((500m cpu req + 50m cpu req) /2) * X% HPA target cpu utilization (((500m cpu req + 50m cpu req) /2) * X% HPA 目标 cpu 利用率
Does it add them?它会添加它们吗? ((500m cpu req + 50m cpu req) * X% HPA target cpu utilization ((500m cpu req + 50m cpu req) * X% HPA 目标 cpu 利用率
Does it track them individually?它会单独跟踪它们吗? (500m cpu req * X% HPA target cpu utilization = target #1, 50m cpu req * X% HPA target cpu utilization = target #2.) (500m cpu req * X% HPA 目标 cpu 利用率 = 目标 #1,50m cpu req * X% HPA 目标 cpu 利用率 = 目标 #2。)
Question 2.)问题2。)
Given the scenario of a multi-container pod, where 1 container has a defined CPU request and a blank CPU request for the other containers:给定多容器 pod 的场景,其中 1 个容器具有定义的 CPU 请求和其他容器的空白 CPU 请求:
How would Kubernetes Horizontal Pod Autoscaler calculate CPU Utilization for Multi Container pods? Kubernetes Horizontal Pod Autoscaler 如何计算多容器 Pod 的 CPU 利用率?
Does it work as if you only had a 1 container pod?它是否像只有 1 个容器 pod 一样工作?
Question 3.)问题 3。)
Do the answers to questions 1 and 2 change based on the HPA API version?问题 1 和 2 的答案是否会根据 HPA API 版本而变化?
I noticed stable/nginx-ingress helm chart, chart version 1.10.2, deploys an HPA for me with these specs:我注意到 stable/nginx-ingress helm chart,图表版本 1.10.2,为我部署了一个具有以下规格的 HPA:
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
(I noticed apiVersion: autoscaling/v2beta2 now exists) (我注意到 apiVersion: autoscaling/v2beta2 现在存在)
Background Info:背景资料:
I recently had an issue with unexpected wild scaling / constantly going back and forth between min and max pods after adding a sidecar(2nd container) to an nginx ingress controller deployment (which is usually a pod with a single container).我最近遇到了一个问题,即在将 sidecar(第二个容器)添加到 nginx 入口控制器部署(通常是一个带有单个容器的 pod)后,在最小和最大 pod 之间不断地来回扩展。 In my case, it was an oauth2 proxy, although I image istio sidecar container folks might run into this sort of problem all the time as well.就我而言,它是一个 oauth2 代理,尽管我认为 istio sidecar 容器的人也可能一直遇到此类问题。
apiVersion: apps/v1
kind: Deployment
spec:
replicas: 3
template:
spec:
containers:
- name: nginx-ingress-controller #(primary-container)
resources:
requests:
cpu: 500m #baseline light load usage in my env
memory: 2Gi #according to kubectl top pods
limits:
memory: 8Gi #(oom kill pod if this high, because somethings wrong)
- name: oauth2-proxy #(newly-added-2nd-sidecar-container)
resources:
requests:
cpu: 50m
memory: 50Mi
limits:
memory: 4Gi
I have an HPA (apiVersion: autoscaling/v1) with:我有一个 HPA (apiVersion: autoscaling/v1):
It occurred to me that my misconfiguration leads to unexpected wild scaling was caused by 2 issues:我突然想到,我的错误配置导致意外的疯狂扩展是由两个问题引起的:
To address the first issue: I brainstormed my understanding of how it works in the single container scenario (and then realized I don't know the multi-container scenario so I decided to ask this question)解决第一个问题:我集思广益地理解它在单容器场景中的工作原理(然后意识到我不了解多容器场景所以我决定问这个问题)
This is my understanding of how HPA (autoscaling/v1) works when I have 1 container (temporarily ignore the 2nd container in the above deployment spec):这是我对当我有 1 个容器时 HPA(自动缩放/v1)如何工作的理解(暂时忽略上述部署规范中的第二个容器):
The HPA would spawn more replicas when the CPU utilization average of all pods shifted from my normal expected load of 500m or less to 750m (150% x 500m request)当所有 Pod 的 CPU 平均利用率从我的正常预期负载 500m 或更少转移到 750m(150% x 500m 请求)时,HPA 将产生更多副本
To address the 2nd issue: I found out how to dig to see concrete numeric value-based metrics vs relative percentage-based metrics to help figure out what's happening behind the scenes:为了解决第二个问题:我发现了如何挖掘以查看基于具体数值的指标与基于相对百分比的指标,以帮助找出幕后发生的事情:
bash# kubectl describe horizontalpodautoscaler nginx-ingress-controller -n=ingress | grep Metrics: -A 1
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): 5% (56m) / 100%
(Note: kubectl top pods -n=ingress, showed cpu usage of the 5 replicas as 36m, 34m, 88m, 36m, 91m, so that 57m current which ~matches 56m current) (注意:kubectl top pods -n=ingress,显示 5 个副本的 CPU 使用情况为 36m、34m、88m、36m、91m,因此 57m 电流与 56m 电流匹配)
Also now it's a basic proportions Math Problem that allows solving for target static value:此外,现在它是一个基本的比例数学问题,可以求解目标静态值:
(5% / 56m) = (100% / xm) --> x = 56 * 100 / 5 = 1120m target cpu (5% / 56m) = (100% / xm) --> x = 56 * 100 / 5 = 1120m 目标 cpu
(Note: this HPA isn't associated with the deployment mentioned above, that's why the numbers are off.) (注意:此 HPA 与上述部署无关,这就是数字关闭的原因。)
Basing on stackoverflow community member answer in other case其他情况下基于stackoverflow社区成员的回答
"HPA calculates pod cpu utilization as total cpu usage of all containers in pod divided by total request. I don't think that's specified in docs anywhere, but the relevant code is here " “HPA 将 pod cpu 利用率计算为 pod 中所有容器的总 cpu 使用率除以总请求。我认为文档中没有指定任何地方,但相关代码在这里”
You have got more informations,with examples in the link above.您有更多信息,上面的链接中有示例。
Basing on documentation基于文档
Horizontal Pod Autoscaler automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization (or, with beta support, on some other, application-provided metrics ). Horizontal Pod Autoscaler根据观察到的 CPU 利用率(或者,在测试版支持下,应用程序提供的一些其他指标)自动扩展复制控制器、部署或副本集中的Pod数量。
So basically:所以基本上:
apiVersion autoscaling/v1 HPA base on cpu .基于cpu 的apiVersion autoscaling /v1 HPA。
apiVersion autoscaling/v2beta2 base on cpu,memory,custom metrics . apiVersion 自动缩放/v2beta2 基于cpu、内存、自定义指标。
For question 2 if we look at source code we can see that it looks all containers individually and returns if container doesn't have request.对于问题 2,如果我们查看源代码,我们可以看到它单独查看所有容器,如果容器没有请求则返回。
func calculatePodRequests(pods []*v1.Pod, resource v1.ResourceName) (map[string]int64, error) {
requests := make(map[string]int64, len(pods))
for _, pod := range pods {
podSum := int64(0)
for _, container := range pod.Spec.Containers {
if containerRequest, ok := container.Resources.Requests[resource]; ok {
podSum += containerRequest.MilliValue()
} else {
return nil, fmt.Errorf("missing request for %s", resource)
}
}
requests[pod.Name] = podSum
}
return requests, nil
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.