kubernetes is scheduling pods to master

Question

A few days ago I looked up that some pods are being scheduled to master node.

I want to understand why the cluster works this way by default instead of scheduling to workers nodes.

Answer 1

You should check the Taint/Toleration concept. I think this Kub Docs link describes how pods are un-assigned to specific nodes quite effectively. Eg, the master node holds by default a NoSchedule taint and pods with corresponding toleration can schedule on Master Node.

Unless, pods were created with Node Affinity to schedule on a specific node. The same link again refers to Node Affinity as well. In both cases, pods were customized to hold toleration or affinity or you've customized the master.

Use kubectl describe command to check the referenced attributes on node or pod.

Answer 2

Like @AYA mentioned, Taint/Toleration, NodeSelector, Affinity are correct ways to prevent schedule pods on node.

However, in cluster created by kubeadm, master node already have taint to prevent random pods running on the master node. The exception is made for pods with corresponding toleration included in the specification. You did not provide YAML of the pod that has been scheduled on the master. Reason why K8s scheduling pods to MasterNode might be related to kubernetes networking or kubernetes control-plan like flannel, calico, wave, etc.

To restore default taint to the master node you have to execute command below:

$ kubectl taint nodes <master-node-name> node-role.kubernetes.io/master="":NoSchedule

Please keep in mind that some pods are designed to run on master node and we should not prevent them from scheduling. It might affect cluster and may result with poor performance or unstable work.

On master node should be at least:

• storage-provisioner
• kube-scheduler
• kube-controller-manager
• kube-apiserver
• etcd