Openshift 上的 SSL/TLS 直通 NGINX-Ingress-Controller 不工作

Question

I have deployed NGINX-Operator and NGINX-Ingress-Controller per the following github and the secrets from devopscube .

The current setup is:

AWS Classic LB -> ROSA Cluster [Helm NGINX-Ingress-Controller -> NGINX-Ingress -> Service -> Pod]

Here is the YAML file I used to create the NGINX-Ingress-Controller Resource. You will see that enableTLSPassthrough is set to true. However, I am unsure this is taking effect. My goal here is end to end TLS encryption from client to the NGINX service/pod. Right now I am met with error code 400 when accessing in browser through http (http works perfect fine in hello-world set up).

"400 Bad Request The plain HTTP request was sent to HTTPS port"

kind: NginxIngress
apiVersion: charts.nginx.org/v1alpha1
metadata:
  name: nginxingress
  namespace: nginx-ingress
spec:
  controller:
    affinity: {}
    appprotect:
      enable: false
    appprotectdos:
      debug: false
      enable: false
      maxDaemons: 0
      maxWorkers: 0
      memory: 0
    config:
      annotations: {}
      entries: {}
    customPorts: []
    defaultTLS:
      secret: nginx-ingress/default-server-secret
    enableCertManager: false
    enableCustomResources: true
    enableExternalDNS: false
    enableLatencyMetrics: false
    enableOIDC: false
    enablePreviewPolicies: false
    enableSnippets: false
    enableTLSPassthrough: true
    extraContainers: []
    globalConfiguration:
      create: false
      spec: {}
    healthStatus: false
    healthStatusURI: /nginx-health
    hostNetwork: false
    image:
      pullPolicy: IfNotPresent
      repository: nginx/nginx-ingress
      tag: 2.3.0-ubi
    ingressClass: nginx
    initContainers: []
    kind: deployment
    logLevel: 1
    nginxDebug: false
    nginxReloadTimeout: 60000
    nginxStatus:
      allowCidrs: 127.0.0.1
      enable: true
      port: 8080
    nginxplus: false
    nodeSelector: {}
    pod:
      annotations: {}
      extraLabels: {}
    priorityClassName: null
    readyStatus:
      enable: true
      port: 8081
    replicaCount: 1
    reportIngressStatus:
      annotations: {}
      enable: true
      enableLeaderElection: true
      ingressLink: ''
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
    service:
      annotations: {}
      create: true
      customPorts: []
      externalIPs: []
      externalTrafficPolicy: Local
      extraLabels: {}
      httpPort:
        enable: true
        nodePort: ''
        port: 80
        targetPort: 80
      httpsPort:
        enable: true
        nodePort: ''
        port: 443
        targetPort: 443
      loadBalancerIP: ''
      loadBalancerSourceRanges: []
      type: LoadBalancer
    serviceAccount:
      imagePullSecretName: ''
    setAsDefaultIngress: true
    terminationGracePeriodSeconds: 30
    tolerations: []
    volumeMounts: []
    volumes: []
    watchNamespace: ''
    wildcardTLS:
      secret: null
  nginxServiceMesh:
    enable: false
    enableEgress: false
  prometheus:
    create: true
    port: 9113
    scheme: http
    secret: ''
  rbac:
    create: true

Taking a look at the NGINX-Ingress-Controller pod logs on creation I can see nothing about TLS being enabled. A flag does get set in the args section once the pod deploys but I am still unsure this is working.

W0802 20:33:26.594545       1 flags.go:273] Ignoring unhandled arguments: []
I0802 20:33:26.594683       1 flags.go:190] Starting NGINX Ingress Controller Version=2.3.0 PlusFlag=false
I0802 20:33:26.594689       1 flags.go:191] Commit=979db22d8065b22fedb410c9b9c5875cf0a6dc66 Date=2022-07-12T08:51:24Z DirtyState=false Arch=linux/amd64 Go=go1.18.3
I0802 20:33:26.601340       1 main.go:210] Kubernetes version: 1.22.0
I0802 20:33:26.606551       1 main.go:326] Using nginx version: nginx/1.23.0
2022/08/02 20:33:26 [notice] 13#13: using the "epoll" event method
2022/08/02 20:33:26 [notice] 13#13: nginx/1.23.0
2022/08/02 20:33:26 [notice] 13#13: built by gcc 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC) 
2022/08/02 20:33:26 [notice] 13#13: OS: Linux 4.18.0-305.19.1.el8_4.x86_64
2022/08/02 20:33:26 [notice] 13#13: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2022/08/02 20:33:26 [notice] 13#13: start worker processes
2022/08/02 20:33:26 [notice] 13#13: start worker process 15
2022/08/02 20:33:26 [notice] 13#13: start worker process 16
2022/08/02 20:33:26 [notice] 13#13: start worker process 17
2022/08/02 20:33:26 [notice] 13#13: start worker process 18
I0802 20:33:26.630298       1 listener.go:54] Starting Prometheus listener on: :9113/metrics
I0802 20:33:26.630860       1 leaderelection.go:248] attempting to acquire leader lease nginx-ingress/nginxingress-nginx-ingress-leader-election...
I0802 20:33:26.639466       1 leaderelection.go:258] successfully acquired lease nginx-ingress/nginxingress-nginx-ingress-leader-election

Here is the Ingress Resource YAML

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  annotations:
#    kubernetes.io/ingress.class: addon-http-application-routing

#   nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
#    nginx.ingress.kubernetes.io/ssl-redirect: "true"
#    nginx.ingress.kubernetes.io/proxy-redirect-from: https
#    nginx.ingress.kubernetes.io/proxy-redirect-to: https
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
#    nginx.ingress.kubernetes.io/proxy-ssl-protocols: "HTTPS"
#    nginx.ingress.kubernetes.io/secure-backends: "true"
spec:
  defaultBackend:
    service:
      name: nginx
      port:
        number: 443
  ingressClassName: nginx
  tls:
   - hosts:
       - nginx-tlssni.apps.clustername.openshiftapps.com
     secretName: nginx-tls
  rules:
  - host: "nginx-tlssni.apps.clustername.openshiftapps.com"
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: nginx
            port:
              number: 443

Thank you for your insight:)

Answer 1

there are many kinds of NGINX based ingress controllers. The two that are most easily confused are the NGINX INC ingress controller , and the CNCF Kubernetes Ingress Controller .

My understanding is that nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" is for the CNCF Kubernetes Ingress Controller.

Now to your question - based on this example , try changing your annotations to the following:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  annotations:
    nginx.org/ssl-services: "nginx" # Name of your k8s service with TLS
  ...