Kubernetes 入口 TLS 未使用无头服务创建

Question

我想要达到的目标

我正在尝试使用 let's encrypt 在带有 TLS 的 microk8s 集群命名空间中部署 elixir (phoenix) 应用程序。 该集群托管在 AWS EC2 实例上。

我面临的问题

• 在命名空间中创建入口
• 到正确域的入口路由
• 应用程序正在运行并显示在给定的域上

没有在命名空间中创建 TLS 密钥，而是创建了一个“默认”密钥

部署phoenix app和httpbin app后的秘密：

me@me:~/Documents/kubernetes-test$ kubectl get secret -n production
NAME                           TYPE                                  DATA   AGE
default-token-jmgrg            kubernetes.io/service-account-token   3      20m
httpbin-tls                    kubernetes.io/tls                     2      81s

域不安全，即 TLS 不工作。

应用 yml 文件后来自入口 controller 的日志：

W0106 17:26:36.967036       6 controller.go:1192] Error getting SSL certificate "production/phoenix-app-tls": local SSL certificate production/phoenix-app-tls was not found. Using default certificate
W0106 17:26:46.445248       6 controller.go:1192] Error getting SSL certificate "production/phoenix-app-tls": local SSL certificate production/phoenix-app-tls was not found. Using default certificate
W0106 17:26:49.779680       6 controller.go:1192] Error getting SSL certificate "production/phoenix-app-tls": local SSL certificate production/phoenix-app-tls was not found. Using default certificate
I0106 17:26:56.431925       6 status.go:281] "updating Ingress status" namespace="production" ingress="phoenix-app-ingress" currentValue=[] newValue=[{IP:127.0.0.1 Hostname: Ports:[]}]
I0106 17:26:56.443405       6 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"production", Name:"phoenix-app-ingress", UID:"REDACTED", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"1145907", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
W0106 17:26:56.443655       6 backend_ssl.go:46] Error obtaining X.509 certificate: no object matching key "production/phoenix-app-tls" in local store
W0106 17:26:56.443781       6 controller.go:1192] Error getting SSL certificate "production/phoenix-app-tls": local SSL certificate production/phoenix-app-tls was not found. Using default certificate

创建入口的描述，请注意，在底部它说Successfully created Certificate "phoenix-app-tls" but the secret does not exist ：

me@me:~/Documents/kubernetes-test$ kubectl describe ing phoenix-app-ingress -n production
Name:             phoenix-app-ingress
Labels:           app=phoenix-app
Namespace:        production
Address:          127.0.0.1
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  phoenix-app-tls terminates phoenix.sub.mydomain.com
Rules:
  Host                           Path  Backends
  ----                           ----  --------
  phoenix.sub.mydomain.com  
                                 /   phoenix-app-service-headless:8000 (REDACTED_IP:4000,REDACTED_IP:4000)
Annotations:                     cert-manager.io/cluster-issuer: letsencrypt
                                 nginx.ingress.kubernetes.io/cors-allow-credentials: true
                                 nginx.ingress.kubernetes.io/cors-allow-methods: GET, POST, OPTIONS
                                 nginx.ingress.kubernetes.io/cors-allow-origin: *
                                 nginx.ingress.kubernetes.io/enable-cors: true
Events:
  Type    Reason             Age                  From                      Message
  ----    ------             ----                 ----                      -------
  Normal  CreateCertificate  29m                  cert-manager              Successfully created Certificate "phoenix-app-tls"
  Normal  Sync               8m43s (x3 over 29m)  nginx-ingress-controller  Scheduled for sync

资源

部署 yml：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: phoenix-app
  labels:
    app: phoenix-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: phoenix-app
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: phoenix-app
    spec:
      containers:
      - name: phoenix-app
        image: REDACTED
        imagePullPolicy: Always
        command: ["./bin/hello", "start"]
        lifecycle:
          preStop:
            exec:
              command: ["./bin/hello", "stop"]
        ports:
        - containerPort: 4000
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        envFrom:
        - configMapRef:
            name: phoenix-app-config
        - secretRef:
            name: phoenix-app-secrets
      imagePullSecrets:
      - name: gitlab-pull-secret

服务 yml：

apiVersion: v1
kind: Service
metadata:
  name: phoenix-app-service-headless
  labels:
    app: phoenix-app
spec:
  clusterIP: None
  selector:
    app: phoenix-app
  ports:
  - name: http
    port: 8000
    targetPort: 4000 # The exposed port by the phoenix app

注意：我删除了我的实际域

入口 yml：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: phoenix-app-ingress
  labels:
    app: phoenix-app
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, OPTIONS"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
    cert-manager.io/cluster-issuer: "letsencrypt"
spec:
  tls:
  - hosts:
    - "phoenix.sub.mydomain.com"
    secretName: phoenix-app-tls
  rules:
  - host: "phoenix.sub.mydomain.com"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: phoenix-app-service-headless
            port:
              number: 8000 # Same port as in service.yml

用不同的服务测试

我使用 httpbin 部署了一个示例服务（不是无头服务），并且 TLS 在同一个命名空间中运行良好。 以下是我用来部署它的资源：

部署.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  replicas: 1
  selector: 
    matchLabels:
      app: httpbin
      version: v1
  template:
    metadata:
      labels:
        app: httpbin
        version: v1
    spec:
      containers:
      - image: docker.io/kennethreitz/httpbin
        imagePullPolicy: Always
        name: httpbin
        ports:
        - containerPort: 80

服务 yml：

apiVersion: v1
kind: Service
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  ports:
  - name: http
    port: 8000
    targetPort: 80
  selector:
    app: httpbin

入口 yml：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin
  labels:
    app: httpbin
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt"
spec:
  tls:
  - hosts:
    - "httpbin.sub.mydomain.com"
    secretName: httpbin-tls
  rules:
  - host: "httpbin.sub.mydomain.com" # This is a subdomain we want to route these requests to
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port:
              number: 8000

我最好的猜测是它与服务是无头的事实有关，但我不知道如何解决这个问题。

Answer 1

我发现您实际上可以使用 kubectl 检查证书： kubectl get certificate -n production

此证书的状态为 READY = FALSE。

我检查了描述： kubectl describe certificate <certificate_name> -n production

在底部它说：在过去 164 小时内为这个确切的域创建了太多证书。

我刚刚更改了域，瞧。 有用。