k8s pod 以在 EKS 中的现货和按需实例中进行调度

Question

we are planning to introduce AWS spot instances in production ( non-prod is running with spot already ). In order to achieve HA we are running HPA with minimum replicas 2 for all critical deployments. Because of the spot instances behaviour we want to run on-demand instances and one pod should be running on on-demand instances for the same

Question:

Is there anyway i can split pods to get launch one pod of the deployment in on-demand and all the other pods (another one since minimum is 2 and if HPA increase the pods ) of the same deployment in spot instances.

We already using nodeaAffinity and podAntiAffinity since we have multiple node groups for different reasons. Below is the snippet.

        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: category
                operator: In
                values:
                - <some value>
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: <lable key>
                operator: In
                values: 
                - <lable value>
            topologyKey: "kubernetes.io/hostname"
    

Answer 1

Following up on your last message:

Will check the two deployments with same label in non-prod then we will update here.

Just wondering how this went. Was there any issues/gotchas from this setup that you could share. Are you currently using this setup, or have you moved on to another setup.

Answer 2

Short answer is No . No such way to define per replica. As you are already using podAntiAffinity, just by adding the same pod labels, you can ensure no two replicas stays in the same host (if that's not what you are already doing). And then use spotInterruption Handler to drain and reschedule without abrupt downtimes during spot interruptions.