TLS 不適用於 Kubernetes 中的 LoadBalancer 支持的服務

Question

我正在嘗試通過將服務類型創建為負載均衡器來公開集群中的應用程序。 這樣做的原因是我希望這個應用程序有一個單獨的通信渠道。 我有一個 KOPS 集群。 我想使用 AWS 的網絡負載均衡器，以便獲得靜態 IP。 當我創建端口 80 映射到應用程序運行的端口的服務時，一切正常，但是當我嘗試添加端口 443 時，它只是超時。

這是有效的配置 -

apiVersion: v1
metadata:
  name: abc
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
  labels:
    app: abc
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 9050
  selector:
    app: abc
  type: LoadBalancer

一旦我在配置文件中添加 TLS 支持並部署它。 與負載平衡器的連接超時。 如何向負載均衡器添加 TLS 支持？ 我想通過服務而不是通過入口來做到這一點。 這是對我不起作用的配置，當我在瀏覽器中粘貼鏈接時，它超時了。

kind: Service
apiVersion: v1
metadata:
  name: abc
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxxx
  labels:
    app: abc
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 443
    protocol: TCP
    targetPort: 9050
  selector:
    app: abc
  type: LoadBalancer

Answer 1

您可以使用 tls & ssl 終止

   apiVersion: v1
    kind: Service
    metadata:
      name: test-service
      annotations:
        # Note that the backend talks over HTTP.
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
        # TODO: Fill in with the ARN of your certificate.
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:{region}:{user id}:certificate/{id}
        # Only run SSL on the port named "https" below.
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    spec:
      selector:
        app: test-pod
      ports:
      - name: http
        port: 80
        targetPort: 8080
      - name: https
        port: 443
        targetPort: 8080
      type: LoadBalancer

您可以在 aws 證書管理器中添加 tls certficate 並使用證書的 arn 地址到 kubernetes 服務。

就像在 becked 中，您可以終止 https 連接並僅使用 HTTP。

你也可以看看這個： https : //aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/

https://github.com/kubernetes/kubernetes/issues/73297

編輯：1

service.beta.kubernetes.io/aws-load-balancer-type: nlb

如果不起作用，請嘗試根據您的負載均衡器類型添加此注釋。

Answer 2

您現在可以使用 NLB 和 SSL 終止（NLB 中的 https > 服務中的 http）部署入口。 終於找到了對我有用的解決方案，您可以嘗試部署以下 ingress.yaml（確保在部署部分更新您的證書 ARN）：

    ---
# Source: nginx-ingress/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
---
# Source: nginx-ingress/templates/default-backend-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-backend
---
# Source: nginx-ingress/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    component: "controller"
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-controller
data:
  server-snippet: |
    listen 8000;
    if ( $server_port = 80 ) {
       return 308 https://$host$request_uri;
    }
  ssl-redirect: "false"
---
# Source: nginx-ingress/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - update
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
---
# Source: nginx-ingress/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress
subjects:
  - kind: ServiceAccount
    name: nginx-ingress
    namespace: ingress-nginx
---
# Source: nginx-ingress/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - update
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader-nginx
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - create
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
---
# Source: nginx-ingress/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress
subjects:
  - kind: ServiceAccount
    name: nginx-ingress
    namespace: ingress-nginx
---
# Source: nginx-ingress/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:##REPLACE WITH YOUR CERT ARN"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    component: "controller"
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-controller
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: special
  selector:
    app: nginx-ingress
    release: nginx-ingress
    app.kubernetes.io/component: controller
  type: "LoadBalancer"
---
# Source: nginx-ingress/templates/default-backend-service.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    component: "default-backend"
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-default-backend
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
  selector:
    app: nginx-ingress
    release: nginx-ingress
    app.kubernetes.io/component: default-backend
  type: "ClusterIP"
---
# Source: nginx-ingress/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
    app.kubernetes.io/component: controller
  name: nginx-ingress-controller
  annotations:
    {}
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: nginx-ingress
  replicas: 1
  revisionHistoryLimit: 10
  strategy:
    {}
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app: nginx-ingress
        release: nginx-ingress
        component: "controller"
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirst
      containers:
        - name: nginx-ingress-controller
          image: "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0"
          imagePullPolicy: "IfNotPresent"
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/nginx-ingress-default-backend
            - --publish-service=$(POD_NAMESPACE)/nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/nginx-ingress-controller
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            capabilities:
                drop:
                - ALL
                add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: special
              containerPort: 8000
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          resources:
            {}
      hostNetwork: false
      serviceAccountName: nginx-ingress
      terminationGracePeriodSeconds: 60
---
# Source: nginx-ingress/templates/default-backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
    app.kubernetes.io/component: default-backend
  name: nginx-ingress-default-backend
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: nginx-ingress
  replicas: 1
  revisionHistoryLimit: 10
  template:
    metadata:
      labels:
        app: nginx-ingress
        release: nginx-ingress
        app.kubernetes.io/component: default-backend
    spec:
      containers:
        - name: nginx-ingress-default-backend
          image: "k8s.gcr.io/defaultbackend-amd64:1.5"
          imagePullPolicy: "IfNotPresent"
          args:
          securityContext:
            runAsUser: 65534
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 0
            periodSeconds: 5
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 6
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          resources:
            {}
      serviceAccountName: nginx-ingress-backend
      terminationGracePeriodSeconds: 60

Answer 3

您的注釋是指https端口

service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"

但是您的端口名為http ，請更改為https

spec:
  externalTrafficPolicy: Local
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 9050