Balancing traffic using least connection in Kubernetes

Question

I have a Kubernetes cluster with a deployment like the next one:

The goal here is to deploy an application in multiple pods exposed through a ClusterIP service named my-app . The same deployment is made in multiple namespaces (A, B and C), changing slightly the config of the application. Then, in some nodes I have an HAProxy using hostNetwork to bind to the node ports. These HAProxy are exposed to my clients through a DNS pointing to them (my_app.com).

When a client connects to my app, they send a header specifying the namespace to which the request should be redirected (A, B or C) and the HAProxy resolves the IP of the service using do-resolve against a dns entry like my_app.A.svc.cluster.local , which returns the IP of the service my_app in the namespace A . That way I can have a single entry point (single DNS record) and a single port (80) to my cluster, which is one of my requirements. I'm also able to create new namespaces and deploy other configs of my app without the need to modify the HAProxies, which is the second requirement.

Now, the requests I get are a mix of short and long requests so I need to use least connection here. This is is not possible in the HAProxies as I don't have a list of backends (the redirection is dynamic as you can see in the code below). I'm trying to use kube-proxy with IPVS and least connection mode. What I noticed is that the tracking of connections to the different pods is per node, and this information is not shared between the different nodes. This way, if two request to my_app.com Namespace: A are processed by two different nodes, both can go to the same pod (eg. pod_1) as in each node, the number of active connections to that pod is 0. The problem becomes worse as I increase the number of HAProxies behind the DNS.

How can I solve this problem and have a better balance without having a single entry point to the cluster (having a single HAProxy behind the DNS)?

I'm adding here the code used in HAProxy to route based on headers:

resolvers dns
    hold nx 3s
    hold other 3s
    parse-resolv-conf

frontend my_app_frontend
    bind :80
    default_backend my_app_backend
    http-request set-var(sess.namespace) hdr(X-Namespace)
    http-request do-resolve(txn.service,dns,ipv4) str(),concat(my_app.,sess.namespace,.svc.cluster.local)

backend my_app_backend
    http-request set-dst var(txn.service)
    http-request set-dst-port int(80)
    server service 0.0.0.0:0

Answer 1

I would use the peers feature from HAProxy to save the sessions for the namespaces cross nodes border.
https://www.haproxy.com/blog/introduction-to-haproxy-stick-tables/

In short and untested

peers mypeers
  peer node1 192.168.122.64:10000
  peer node2 192.168.122.1:10000

backend my_app_backend
  stick-table type string len 32 size 100k expire 30m peers mypeers
  stick on hdr(X-Namespace)
  http-request set-dst var(txn.service)
  http-request set-dst-port int(80)
  server service 0.0.0.0:0