為什么 Kubernetes HPA 縮放不縮小（內存）？

Question

概括

在我們的 Kubernetes 集群中，我們引入了帶有 memory 和 CPU 限制的 HPA。 現在我們不明白為什么我們有一個服務的 2 個副本。

有問題的服務使用 57% / 85% Memory 並且有 2 個副本而不是 1 個。 我們認為這是因為當您將兩個吊艙的 memory 相加時，它超過了 85%，但如果只有一個吊艙，則不會。 那么這是否會阻止它縮小規模？ 我們可以在這里做什么？

當我們部署服務時，我們還觀察到 memory 的使用量達到峰值。 我們在 aks (azure) 中使用 spring-boot 服務，並認為它可能會在那里擴大規模並且永遠不會下降。 我們錯過了什么或有任何建議嗎？

舵

帕：

{{- $fullName := include "app.fullname" . -}}
{{- $ := include "app.fullname" . -}}

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ $fullName }}-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "app.name" . }}
  minReplicas: 1
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 50
    - type: Resource
      resource:
        name: memory
        targetAverageUtilization: 85

並在部署中：

# Horizontal-Pod-Auto-Scaler
          resources:
            requests:
              memory: {{ $requestedMemory }}
              cpu: {{ $requesteCpu }}
            limits:
              memory: {{ $limitMemory }}
              cpu: {{ $limitCpu }}

使用服務默認值：

hpa:
  resources:
    request:
      memory: 500Mi
      cpu: 300m
    limits:
      memory: 1000Mi
      cpu: 999m

kubectl 獲取 hpa -n dev

NAME                            REFERENCE                              TARGETS           MINPODS   MAXPODS   REPLICAS   AGE
xxxxxxxx-load-for-cluster-hpa   Deployment/xxxxxxxx-load-for-cluster   34%/85%, 0%/50%   1         10        1          4d7h
xxx5-ccg-hpa                    Deployment/xxx5-ccg                    58%/85%, 0%/50%   1         10        1          4d12h
iotbootstrapping-service-hpa    Deployment/iotbootstrapping-service    54%/85%, 0%/50%   1         10        1          4d12h
mocks-hpa                       Deployment/mocks                       41%/85%, 0%/50%   1         10        1          4d12h
user-pairing-service-hpa        Deployment/user-pairing-service        41%/85%, 0%/50%   1         10        1          4d12h
aaa-registration-service-hpa    Deployment/aaa-registration-service    57%/85%, 0%/50%   1         10        2          4d12h
webshop-purchase-service-hpa    Deployment/webshop-purchase-service    41%/85%, 0%/50%   1         10        1          4d12h

kubectl 描述 hpa -n dev

Name:                                                     xxx-registration-service-hpa
Namespace:                                                dev
Labels:                                                   app.kubernetes.io/managed-by=Helm
Annotations:                                              meta.helm.sh/release-name: vwg-registration-service
                                                          meta.helm.sh/release-namespace: dev
CreationTimestamp:                                        Thu, 18 Jun 2020 22:50:27 +0200
Reference:                                                Deployment/xxx-registration-service
Metrics:                                                  ( current / target )
  resource memory on pods  (as a percentage of request):  57% (303589376) / 85%
  resource cpu on pods  (as a percentage of request):     0% (1m) / 50%
Min replicas:                                             1
Max replicas:                                             10
Deployment pods:                                          2 current / 2 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from memory resource utilization (percentage of request)
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:           <none>

如果需要任何進一步的信息，請隨時詢問！

非常感謝您抽出寶貴的時間！

干杯羅賓

Answer 1

確定所需副本數的公式是：

desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]

對於您的問題，重要的部分是ceil[...] function 包裝器：它總是四舍五入到下一個最近的副本。 如果currentReplicas為 2 且desiredMetricValue為 85%，則currentMetricValue必須為 42.5% 或更低才能觸發縮減。

在您的示例中， currentMetricValue為 57%，因此您得到

desiredReplicas = ceil[2 * (57 / 85)]
                = ceil[2 * 0.671]
                = ceil[1.341]
                = 2

沒錯，如果currentReplicas為 1，HPA 也不會覺得需要擴大規模； 實際利用率需要攀升至 85% 以上才能觸發。