“ERR_TOO_MANY_REDIRECTS”nginx-ingress controller 不会覆盖 X-Forwarded-Proto：http，X-Forwarded-Scheme：http

Question

我们正在使用 IAP，GCP L7 负载均衡器和 nginx-ingress controller（版本 0.49.3）。 我们部署了自托管 GitLab，我们收到“ERR_TOO_MANY_REDIRECTS”。

经过几天紧张的故障排除后，我们注意到

POST /api/v4/jobs/request HTTP/1.1
Host: gitlab.ci.g.nakhoda.ai
X-Request-ID: dfa874d97ed06e0e7a7cf17c0a4ae2c0
X-Real-IP: 35.191.12.183
X-Forwarded-For: 35.191.12.183
X-Forwarded-Host: gitlab.ci.g.nakhoda.ai
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Scheme: http
X-Scheme: http
X-Original-Forwarded-For: 62.189.73.245, 35.186.225.221
Content-Length: 714
User-Agent: gitlab-runner 15.6.1 (15-6-stable; go1.18.8; linux/amd64)
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip
X-Cloud-Trace-Context: 4efdd65224a5882999fd0fb26a888bfd/2273898499790060164
Via: 1.1 google

X-Forwarded-Proto: http 和 X-Forwarded-Scheme: http 。 在摆弄了一下之后，我们在 nginx-ingress-controller 的容器中执行并编辑nginx.conf为它们都设置了 https 并添加了注释ssl-redirect: false 。

之后请求看起来像这样：

POST /api/v4/jobs/request HTTP/1.1
Host: gitlab.ci.g.nakhoda.ai
X-Request-ID: f103bba96d47527dae15087d1dd1d476
X-Real-IP: 35.191.12.180
X-Forwarded-For: 35.191.12.180
X-Forwarded-Host: gitlab.ci.g.nakhoda.ai
X-Forwarded-Port: 80
X-Forwarded-Proto: https
X-Forwarded-Scheme: https
X-Scheme: http
X-Original-Forwarded-For: 194.203.216.4, 35.186.225.221
Content-Length: 714
User-Agent: gitlab-runner 15.6.1 (15-6-stable; go1.18.8; linux/amd64)
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip
X-Cloud-Trace-Context: 3ceb1d9d95fa28be4da929dea1f3ac95/3016269448751760533
Via: 1.1 google

通过手动编辑，我们现在可以毫无问题地访问 GitLab，但问题是这是我们添加的手动修复，因此每次管道进行部署时，都会将其重置到位。

问题是我们希望添加自定义标头以覆盖 [nginx.conf]( https://kube.netes.github.io/ingress-nginx/examples/customization/custom-headers/不会覆盖正常的 nginx。 conf 配置），但是在添加具有特定内容的 ConfigMap 之后，或者只是将类似的注释添加到 nginx-ingress controller 的 Ingress 或 ConfigMap 中：

Gitlab 入口

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gitlab-chart-webservice-default
  namespace: gitlab
  labels:
    app: webservice
    app.kubernetes.io/managed-by: Helm
    chart: webservice-6.6.2
    gitlab.com/webservice-name: default
    heritage: Helm
    release: gitlab-chart
  annotations:
    kubernetes.io/ingress.provider: nginx
    meta.helm.sh/release-name: gitlab-chart
    meta.helm.sh/release-namespace: gitlab
    nginx.ingress.kubernetes.io/proxy-body-size: 512m
    nginx.ingress.kubernetes.io/proxy-connect-timeout: '15'
    nginx.ingress.kubernetes.io/proxy-read-timeout: '600'
    nginx.ingress.kubernetes.io/service-upstream: 'true'
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'
status:
  loadBalancer:
    ingress:
      - ip: 1.1.1.1
      - ip: 1.1.1.1
spec:
  ingressClassName: stable-protected
  tls:
    - hosts:
        - gitlab.ci.example.com
      secretName: gitlab-cert
  rules:
    - host: gitlab.ci.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: gitlab-chart-webservice-default
                port:
                  number: 8181

ConfigMap nginx controller

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx
  namespace: ingress-stable-protected
  annotations:
    refresh-me: |
      This 'refresh-me' annotation is purely used as a placeholder so we can
      modify its value in order to manually force the nginx ingress-controller
      to reload its configuration.
      Reload.
data:
  enable-vts-status: 'true'
  hide-headers: Strict-Transport-Security
  hsts: 'true'
  hsts-include-subdomains: 'false'
  hsts-preload: 'false'
  proxy-body-size: 2048m
  proxy-buffer-size: 16k
  proxy-connect-timeout: '5'
  proxy-next-upstream: 'off'
  proxy-read-timeout: '600'
  proxy-send-timeout: '600'
  server-name-hash-bucket-size: '256'
  server-tokens: 'false'

nginx.ingress.kubernetes.io/x-forwarded-proto=https
nginx.ingress.kubernetes.io/x-forwarded-scheme=https

它仍然需要 nginx.conf http所以我们有点坚持这个临时修复。

也尝试添加一堆自定义标头和 x-forwarded-headers 但无济于事。

Answer 1

问题是在 nginx-ingress 的 0.24.0 版本之后，注释发生了变化，因此通过将以下内容添加到 nginx configmap 中，不再有错误。

可以在这张票中找到更深入的描述

use-forwarded-headers: "true"
use-proxy-protcol: "false"

完整示例：

apiVersion: v1
data:
  enable-vts-status: "true"
  hide-headers: Strict-Transport-Security
  hsts-preload: "false"
  proxy-body-size: 2048m
  proxy-buffer-size: 16k
  proxy-connect-timeout: "5"
  proxy-next-upstream: "off"
  proxy-read-timeout: "600"
  proxy-send-timeout: "600"
  server-name-hash-bucket-size: "256"
  server-tokens: "false"
  use-forwarded-headers: "true"
  use-proxy-protcol: "false"
kind: ConfigMap
metadata:
  annotations:
    refresh-me: |
      This 'refresh-me' annotation is purely used as a placeholder so we can
      modify its value in order to manually force the nginx ingress-controller
      to reload its configuration.
  labels:
    app.kubernetes.io/managed-by: pulumi
  name: nginx
  namespace: namespace