Using Session Affinity (Cookies) with SSL Passthrough on NGINX-Ingress

Question

TL;DR : I want to setup cookie-based session affinity in K8s over the nginx-ingress controller with SSL passthrough - can this be done?

---

Hey all,

I have a working Azure Kubernetes Service (AKS) running (1.11.3) and have configured the NGINX-Ingress controller to route requests to a ClusterIP Service for my app (which has a minimum of 2 pods running).

I have successfully configured SSL passthrough on the ingress controller so that the TLS is terminated at the Pods and so I can use HTTP2 (as per this article ). Now I'd like to setup Session Affinity (using Cookies) so that connections are routed to the same pod for stateful behaviours (logging into to the app).

I have tried using the following annotations on the ingress object for this:

nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: "route"
nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"

However, I'm not seeing the "route" cookie returned on the first request. I've worked through the issue described here and ensured the ingress is setup correctly. Then I've spotted this message over at the docs :

Because SSL Passthrough works on layer 4 of the OSI model (TCP) and not on the layer 7 (HTTP), using SSL Passthrough invalidates all the other annotations set on an Ingress object.

Q : Would this mean that using session affinity with SSL passthrough is off the table? In that the Ingress isn't going to be able to identify the connection / cookie (as it's SSL encrypted) and direct it to the previously associated pod?

Answer 1

Short answer: no it's impossible. Layer 4 has no clue on what http is, it just sees bytes flowing back and forth. You could have affinity based on ip address instead, just not with cookies as it would need a layer 7 proxy solution. Depending on your situation, you could run a proxy in layer 7 which would be able to decrypt the traffic and then encrypt it with another certificate for internal use. All of the payload (minus SNI for example) is not encrypted as per SSL, which means that in order to do some kind of affinity on cookies, the proxy will need to decrypt the data before inspecting it.