AWS上的Kubernetes：使用Nginx-ingress + cert-manager保留客户端IP

Question

We have setup Kubernetes with nginx-ingress combined with cert-manager to automatically obtain and use SSL certificates for ingress domains using LetsEncrypt using this guide: https://medium.com/@maninder.bindra/auto-provisioning-of-letsencrypt-tls-certificates-for-kubernetes-services-deployed-to-an-aks-52fd437b06b0 . The result is that each Ingress defines its own SSL certificate that is automatically provisioned by cert-manager .

This all works well but for one problem, the source IP address of the traffic is lost to applications in Pods.

There is an annotation that is advised to use to apply to the nginx-ingress controller service service.beta.kubernetes.io/aws-load-balancer-backend-protocol: '*' . This has the effect of preserving source IP addresses. However, doing it breaks SSL:

An error occurred during a connection to {my.domain.com}. SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG

My head is starting to spin. Does anyone know of any approaches to this (it seems to me that this would be a common requirement)?

Ingress configuration:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-http-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
spec:
  rules:
  - host: my.host.com
    http:
      paths:
      - path: /
        backend:
          serviceName: my-http-service
          servicePort: 80
  tls:
  - hosts:
    - "my.host.com"
    secretName: malcolmqa-tls

Answer 1

As @dom_watson mentioned in the comments, adding parameter controller.service.externalTrafficPolicy=Local to Helm install configuration solved the issue due to the fact that Local value preserves the client source IP, thus the network traffic will reach target Pod in Kubernetes cluster. Find more information in the official Kubernetes guidelines .

Answer 2

helm upgrade my-nginx stable/nginx-ingress --set rbac.create=true --set controller.service.externalTrafficPolicy=Local