[英]Kubernetes requests not balanced
We've just had an increase in traffic to our kubernetes cluster and I've noticed that of our 6 application pods, 2 of them are seemingly not used very much.我们的 kubernetes 集群的流量刚刚增加,我注意到在我们的 6 个应用程序 pod 中,其中 2 个似乎没有被经常使用。
kubectl top pods
returns the following kubectl top pods
返回以下内容
You can see of the 6 pods, 4 of them are using more than 50% of the CPU (2 vCPU nodes), but two of them aren't really doing much at all.您可以看到 6 个 pod,其中 4 个使用了超过 50% 的 CPU(2 个 vCPU 节点),但其中两个实际上并没有做太多事情。
Our cluster is setup on AWS, using the ALB ingress controller.我们的集群是在 AWS 上设置的,使用 ALB 入口控制器。 The load balancer is configured to use the
Least outstanding requests
rather than Round robin
in an attempt to balance things out a bit more, but we're still seeing this imbalance.负载平衡器被配置为使用
Least outstanding requests
而不是Round robin
,以尝试平衡更多的事情,但我们仍然看到这种不平衡。
Is there any way of determining why this is happening, or if indeed it actually is a problem?有什么方法可以确定为什么会发生这种情况,或者是否确实存在问题? I'm hoping it's normal behaviour rather than an issue but I'd rather investigate it.
我希望这是正常行为而不是问题,但我宁愿调查它。
App deployment config应用部署配置
This is the configuration of the main application pods.这是主要应用程序 pod 的配置。 Nothing fancy really
真的没什么好看的
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "app.fullname" . }}
labels:
{{- include "app.labels" . | nindent 4 }}
app.kubernetes.io/component: web
spec:
replicas: {{ .Values.app.replicaCount }}
selector:
matchLabels:
app: {{ include "app.fullname" . }}-web
template:
metadata:
annotations:
checksum/config: {{ include (print $.Template.BasePath "/config_maps/app-env-vars.yaml") . | sha256sum }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 10 }}
{{- end }}
labels:
{{- include "app.selectorLabels" . | nindent 8 }}
app.kubernetes.io/component: web
app: {{ include "app.fullname" . }}-web
spec:
serviceAccountName: {{ .Values.serviceAccount.web }}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- {{ include "app.fullname" . }}-web
topologyKey: failure-domain.beta.kubernetes.io/zone
containers:
- name: {{ .Values.image.name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- bundle
args: ["exec", "puma", "-p", "{{ .Values.app.containerPort }}"]
ports:
- name: http
containerPort: {{ .Values.app.containerPort }}
protocol: TCP
readinessProbe:
httpGet:
path: /healthcheck
port: {{ .Values.app.containerPort }}
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 5
resources:
{{- toYaml .Values.resources | nindent 12 }}
envFrom:
- configMapRef:
name: {{ include "app.fullname" . }}-cm-env-vars
- secretRef:
name: {{ include "app.fullname" . }}-secret-rails-master-key
- secretRef:
name: {{ include "app.fullname" . }}-secret-db-credentials
- secretRef:
name: {{ include "app.fullname" . }}-secret-db-url-app
- secretRef:
name: {{ include "app.fullname" . }}-secret-redis-credentials
That's a known issue with Kubernetes.这是 Kubernetes 的一个已知问题。
In short, Kubernetes doesn't load balance long-lived TCP connections.简而言之,Kubernetes 不会对长期存在的 TCP 连接进行负载平衡。
This excellent article covers it in details.这篇优秀的文章详细介绍了它。
The load distribution you service is showing complies exactly with the case.您所服务的负载分布与案例完全相符。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.