Kube.netes Pod 反亲和性 - 基于 label 均匀分布 Pod？

Question

We are finding that our Kube.netes cluster tends to have hot-spots where certain nodes get far more instances of our apps than other nodes.

In this case, we are deploying lots of instances of Apache Airflow, and some nodes have 3x more web or scheduler components than others.

Is it possible to use anti-affinity rules to force a more even spread of pods across the cluster?

Eg "prefer the node with the least pods of label component=airflow-web ?"

If anti-affinity does not work, are there other mechanisms we should be looking into as well?

Answer 1

Try adding this to the Deployment/StatefulSet .spec.template :

      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: "component"
                  operator: In
                  values:
                  - airflow-web
              topologyKey: "kubernetes.io/hostname"

Answer 2

Have you tried configuring the kube-scheduler ?

kube-scheduler selects a node for the pod in a 2-step operation:

• Filtering : finds the set of Nodes where it's feasible to schedule the Pod.
• Scoring : ranks the remaining nodes to choose the most suitable Pod placement.

Scheduling Policies : can be used to specify the predicates and priorities that the kube-scheduler runs to filter and score nodes.

kube-scheduler --policy-config-file <filename>

• Sample config file

One of the priorities for your scenario is:

• BalancedResourceAllocation : Favors nodes with balanced resource usage.

Answer 3

The right solution here is pod topology spread constraints: https://kube.netes.io/blog/2020/05/introducing-podtopologyspread/

Anti-affinity only works until each node has at least 1 pod. Spread constraints actually balances based on the pod count per node.