How to fix Kubernetes Ingress Controller cutting off nodes from cluster

Question

I'm having some trouble installing an Ingress Controller in my on-prem cluster (created with Kubespray, running MetalLB to create LoadBalancer.).

I tried using nginx, traefik and kong but all got the same results.

I'm installing my the nginx helm chart using the following values.yaml:

controller:
  kind: DaemonSet
  nodeSelector:
    node-role.kubernetes.io/master: ""
  image:
    tag: 0.23.0
rbac:
  create: true

With command:

helm install --name nginx stable/nginx-ingress --values values.yaml --namespace ingress-nginx

When I deploy the ingress controller in the cluster, a service is created (eg nginx-ingress-controller for nginx). This service is of the type LoadBalancer and gets an external IP.

When this external IP is assigned, the node that's linked to this external IP is lost (status Not Ready). However, when I check this node, it's still running, it's just cut off from the other nodes, it can't even ping them (No route found). When I remove the service (not the rest of the nginx helm chart), everything works and the Ingress works. I also tried installing nginx/traefik/kong without a LoadBalancer using NodePorts or External IPs on the service, but I get the same result.

Does anyone recognize this behaviour? Why does the ingress still work, even when I remove the nginx-ingress-controller service?

Answer 1

After a long search, we finally found a working solution for this problem.

As mentioned by @A_Suh, the pool of IPs that metallb uses, should contain IPs that are currently not used by one of the nodes in the cluster. By adding a new IP range that's also configured in the DHCP server, metallb can use ARP to link one of the IPs to one of the nodes.

For example in my 5 node cluster (kube11-15): When metallb gets the range 10.4.5.200/31 and allocates 10.4.5.200 for my nginx-ingress-controller, 10.4.5.200 is linked to kube12. On ARP requests for 10.4.5.200, all 5 nodes respond with kube12 and trafic will be routed to this node.