k8s/kind：無法通過 gRPC 連接到入口 controller

Question

我在 MacOS 上並使用kind v0.10.0 。 I've followed the instructions given at https://kind.sigs.k8s.io/docs/user/ingress/ for setting up kind with an nginx ingress controller (and a private docker registry). 不幸的是，我無法連接到127.0.0.1:80上的入口 controller 。 我什至嘗試將我的服務、部署和入口移動到與 nginx 入口 controller 相同的命名空間，即namespace=ingress-nginx ，但即使這樣也沒有解決任何問題。

這是我正在嘗試做的事情：根據 gRPC 方法名稱將傳入的 gRPC 消息路由到兩個單獨的后端服務。

這是我正在觀察的：

• 如果我在瀏覽器中打開http://127.0.0.1 ，我會看到由nginx提供的502頁面。

• 我什至已經確認監聽端口 80 的進程是com.docker.backend 。

• 但是，當我嘗試使用 gRPC 客戶端連接到此端口時，它只是掛起/超時：

   % ~/go/bin/grpcurl -plaintext localhost:80 list Failed to dial target host "localhost:80": context deadline exceeded

誰能幫我調試一下？

豆莢

% kubectl get pods -n ingress-nginx                         
NAME                                       READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-b8tkt       0/1     Completed   0          13h
ingress-nginx-admission-patch-xddd7        0/1     Completed   2          13h
ingress-nginx-controller-9ddc777c5-bwnkx   1/1     Running     0          13h
route-guide-a-6fc997cbd-7zgbk              1/1     Running     0          10m
route-guide-a-6fc997cbd-hgpjv              1/1     Running     0          10m
route-guide-b-7b954b77bb-2p8vm             1/1     Running     0          10m
route-guide-b-7b954b77bb-nmdnt             1/1     Running     0          10m

部署

% kubectl get deployment -n ingress-nginx
NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
ingress-nginx-controller   1/1     1            1           13h
route-guide-a              2/2     2            2           10m
route-guide-b              2/2     2            2           10m

服務

% kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.96.230.48    <none>        80:32236/TCP,443:32573/TCP   13h
ingress-nginx-controller-admission   ClusterIP   10.96.79.228    <none>        443/TCP                      13h
route-guide-a                        ClusterIP   10.96.249.171   <none>        10000/TCP                    11m
route-guide-b                        ClusterIP   10.96.143.194   <none>        10000/TCP                    11m

入口

% kubectl get ingress -n ingress-nginx
NAME                  CLASS    HOSTS   ADDRESS     PORTS   AGE
route-guide-ingress   <none>   *       localhost   80      12m

配置文件

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: route-guide-a
  namespace: ingress-nginx
  labels:
    app: route-guide-a
spec:
  replicas: 2
  selector:
    matchLabels:
      app: route-guide-a
  template:
    metadata:
      labels:
        app: route-guide-a

    spec:
      containers:
        - name: route-guide-a
          image: localhost:5000/route_guide:v4
          imagePullPolicy: IfNotPresent
          command: ["/usr/local/go/bin/go", "run", "server/server.go"]
          ports:
            - containerPort: 10000
              name: grpc-port
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: route-guide-b
  namespace: ingress-nginx
  labels:
    app: route-guide-b
spec:
  replicas: 2
  selector:
    matchLabels:
      app: route-guide-b
  template:
    metadata:
      labels:
        app: route-guide-b

    spec:
      containers:
        - name: route-guide-b
          image: localhost:5000/route_guide:v4
          imagePullPolicy: IfNotPresent
          command: ["/usr/local/go/bin/go", "run", "server/server.go"]
          ports:
            - containerPort: 10000
              name: grpc-port
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name

---
apiVersion: v1
kind: Service
metadata:
  name: route-guide-a
  namespace: ingress-nginx
  labels:
    app: route-guide-a
spec:
  # type: NodePort
  selector:
    app: route-guide-a
  ports:
    - port: 10000
      # nodePort: 31397
      targetPort: 10000
      name: grpc-port
      protocol: TCP

---
apiVersion: v1
kind: Service
metadata:
  name: route-guide-b
  namespace: ingress-nginx
  labels:
    app: route-guide-b
spec:
  # type: NodePort
  selector:
    app: route-guide-b
  ports:
    - port: 10000
      # nodePort: 31398
      targetPort: 10000
      name: grpc-port
      protocol: TCP

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: "nginx"
    # nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
  name: route-guide-ingress
  namespace: ingress-nginx
spec:
  rules:
  - http:
      paths: 
        - path: /routeguide.RouteGuide/GetFeature
          backend:
            serviceName: route-guide-a
            servicePort: grpc-port

        - path: /routeguide.RouteGuide/ListFeatures
          backend:
            serviceName: route-guide-a
            servicePort: grpc-port

        - path: /routeguide.RouteGuide/RecordRoute
          backend:
            serviceName: route-guide-b
            servicePort: grpc-port
        
        - path: /routeguide.RouteGuide/RouteChat
          backend:
            serviceName: route-guide-b
            servicePort: grpc-port

        # catch-all route
        - backend:
            serviceName: route-guide-b
            servicePort: grpc-port

正在運行的集群中的入口配置

以下命令的 Output： kubectl get ingress -n ingress-nginx route-guide-ingress -o yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"name":"route-guide-ingress","namespace":"ingress-nginx"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"route-guide-a","servicePort":"grpc-port"},"path":"/routeguide.RouteGuide/GetFeature"},{"backend":{"serviceName":"route-guide-a","servicePort":"grpc-port"},"path":"/routeguide.RouteGuide/ListFeatures"},{"backend":{"serviceName":"route-guide-b","servicePort":"grpc-port"},"path":"/routeguide.RouteGuide/RecordRoute"},{"backend":{"serviceName":"route-guide-b","servicePort":"grpc-port"},"path":"/routeguide.RouteGuide/RouteChat"},{"backend":{"serviceName":"route-guide-a","servicePort":"grpc-port"}}]}}]}}
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
  creationTimestamp: "2021-05-02T02:57:24Z"
  generation: 2
  managedFields:
  - apiVersion: networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
          f:kubernetes.io/ingress.class: {}
          f:nginx.ingress.kubernetes.io/backend-protocol: {}
      f:spec:
        f:rules: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: "2021-05-02T02:57:24Z"
  - apiVersion: networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        f:loadBalancer:
          f:ingress: {}
    manager: nginx-ingress-controller
    operation: Update
    time: "2021-05-02T02:57:44Z"
  name: route-guide-ingress
  namespace: ingress-nginx
  resourceVersion: "12115"
  uid: be3862b4-13a7-4b96-b5f8-399e28bb373c
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: route-guide-a
            port:
              name: grpc-port
        path: /routeguide.RouteGuide/GetFeature
        pathType: ImplementationSpecific
      - backend:
          service:
            name: route-guide-a
            port:
              name: grpc-port
        path: /routeguide.RouteGuide/ListFeatures
        pathType: ImplementationSpecific
      - backend:
          service:
            name: route-guide-b
            port:
              name: grpc-port
        path: /routeguide.RouteGuide/RecordRoute
        pathType: ImplementationSpecific
      - backend:
          service:
            name: route-guide-b
            port:
              name: grpc-port
        path: /routeguide.RouteGuide/RouteChat
        pathType: ImplementationSpecific
      - backend:
          service:
            name: route-guide-a
            port:
              name: grpc-port
        pathType: ImplementationSpecific
status:
  loadBalancer:
    ingress:
    - hostname: localhost

Answer 1

根據kubernetes/ingress-nginx#2444和kubernetes/ingress-nginx#6313可知，Nginx 不能在 80 端口復用 HTTP/1.1 和 HTTP/2。如下圖，80 端口默認沒有啟用 http2。

$ kubectl exec -it -n $INGRESS_CONTROLLER_NS $INGRESS_CONTROLLER_POD -- cat /etc/nginx/nginx.conf
...
server {
    server_name your.domain ;

    listen 80  ;
    listen [::]:80  ;
    listen 443  ssl http2 ;
    listen [::]:443  ssl http2 ;
}
...

因此，一些可能的解決方案可能是：

• 請改用 SSL。
• 使用評論中提到的自定義模板。