在Kubernetes入口之前的Nginx反向代理

Question

We're running a 6-node bare-metal K8s cluster (3-nodes control and 3-nodes compute plane). The ingress is deployed and it's working well for path-based routes. Recently we've built a HA-loadbalancing cluster in front of the Ingress in order to achieve high-availability of the Ingress itself.

+---------------+       +---------+      +---------+
| Loadbalancers |  -->  | Ingress |  --> | Jenkins |
+---------------+       +---------+      +---------+

The loadbalancers basically run nginx in reverse proxy mode with floating IP address that has a DNS A-type record with wildcard:

*.cnk8sv.infra A <private-IP>

However in this setup we are unable to apply virtual-host-based routing. In such a mode the Ingress rules don't catch any traffic. I have a suspicion that the nginx in front of the K8s is dropping some information in the HTTP headers and hence the Ingress controllers are unable to do the host-based routing.

This is the configuration of the nginx loadbalancers:

   location / {
      proxy_pass https://backend;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Proto https;
      proxy_redirect off;
   }

The Ingress object:

spec:
  rules:
  - host: jenkins.cnk8sv.infra
    http:
      paths:
      - backend:
          serviceName: jenkins
          servicePort: 8080
        path: /

As soon as we remove the host filter in the rule the traffic stars flowing (so yes, the backend is up and running). Any help would be greatly appreciated.

Answer 1

解决方法是在nginx配置中将X-Forwarded-Proto值更改为http ，因为我们将Ingress定位在端口80（即HTTP）上。