Understanding actual and virtual subnets in Kubernetes

Question

I'm setting up a self-managed Kubernetes cluster on AWS, using three instances in a VPC with CIDR 10.0.0.0/16 , and a single AWS subnet 10.0.10.0/24 for all instances/nodes.

When it comes to initializing the cluster using Kubeadm , and specifiying the --pod-network-cidr , can this CIDR be anything? Or does it need to align with the AWS subnet that the host instances (control plane and workers) are on?

Same goes for Cilium network, service networks... Can they all be anything, or must it align with the actual host subnet?

I haven't found any good documentation on the possible (and not possible) subnet configurations for self-managed clusters.

Answer 1

• As per Kubernetes Documentation https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network , it is better if host cidr, pod cidr and service cidr do not overlap. All EC2 Instances get IP from the host CIDR. All Pods in Kubernetes will get IP from the Pod CIDR. These Pod IPs are assigned by Cilium and not by Kubernetes since Cilium is responsible for providing the Networking for Kubernetes in your case. All Kubernetes Services get clusterIPs from Service CIDR. These clusterIPs are assigned by Kubernetes.

• As per below kubeadm init documentation, default for service-cidr is 10.96.0.0/12 which is not overlapping with your VPC cidr 10.0.0.0/16

kubeadm init --help | grep cidr
      --pod-network-cidr string              Specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.
      --service-cidr string                  Use alternative range of IP address for service VIPs. (default "10.96.0.0/12")

• As per https://cilium.io/blog/2018/09/26/bionic-beaver#initialize-the-kubernetes-master Blog, you are supposed to use CIDR 10.217.0.0/16 for pods. So please run below command for kubeadm init. CIDR 10.217.0.0/16 also does not overlap with your VPC cidr 10.0.0.0/16

sudo kubeadm init --pod-network-cidr=10.217.0.0/16