KNative 和帶有 nginx-ingress 的 gRPC

Question

我已經在我的 AWS EKS 集群中安裝了 Knative/KServe。 一切正常，但最近我們決定為部署在那里的服務嘗試 gRPC。 它是用 Istio 部署的，winth nginx ingress 在所有東西前面，ingress 指向 Istio ingress gateway：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
  name: computing-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /
        pathType: Prefix

如 Knative 文檔https://github.com/meteatamel/knative-tutorial/blob/master/docs/grpc.md中所述，我通過添加 h2c 端口塊更改了我的 InferenceService yaml：

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  annotations:
  finalizers:
  - inferenceservice.finalizers
  name: triton-test
  namespace: default
spec:
  predictor:
    model:
      args:
      - --model-control-mode=poll
      - --repository-poll-secs=5
      - --allow-grpc=true
      - --grpc-port=9000
      - --log-verbose=0
      env:
      - name: CUDA_VISIBLE_DEVICES
        value: "0"
      - name: S3_DATA_PATH
        value: s3://mymodeldata/
      - name: S3_PARAMS
        value: --region us-east-2 --no-sign-request
      image: XXXXXXX.dkr.ecr.us-east-2.amazonaws.com/ml:mytriton
      modelFormat:
        name: triton
      name: kserve-container
      ports:
      - containerPort: 9000
        name: h2c
        protocol: TCP
      protocolVersion: v2
      resources:
        limits:
          cpu: "2"
          memory: 8Gi
          nvidia.com/gpu: "1"
        requests:
          cpu: "2"
          memory: 8Gi
          nvidia.com/gpu: "1"
      storageUri: s3://mymodeldata/
      volumeMounts:
      - mountPath: /dev/shm
        name: dshm
    nodeSelector:
      DedicatedFor: GPU
    volumes:
    - emptyDir:
        medium: Memory
        sizeLimit: 2Gi
      name: dshm

由於 gRPC 注釋是入口級的，我已經將主入口更改為具有更具體的路徑：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
  name: computing-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /v1
        pathType: Prefix
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /v2
        pathType: Prefix

然后創建第二個入口

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
    nginx.ingress.kubernetes.io/grpc-backend: "true"
  name: computing-grpc-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo
        pathType: ImplementationSpecific

但我無法讓它以任何方式工作。 我嘗試了一百萬種不同的配置，但出現 404 或 502 錯誤。 我的 istio 入口服務是：

apiVersion: v1
kind: Service
metadata:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /healthz/ready
    alb.ingress.kubernetes.io/healthcheck-port: "31619"
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: unknown
    istio: ingressgateway
    istio.io/rev: default
    operator.istio.io/component: IngressGateways
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
spec:
  clusterIP: 172.17.17.19
  clusterIPs:
  - 172.17.17.19
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: status-port
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: http2
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  sessionAffinity: None
  type: ClusterIP

有什么辦法讓它工作嗎？ 我不確定我應該添加哪些附加信息。 謝謝！

Answer 1

由於您將兩個不同的 HTTP 路由器鏈接在一起，您可能想嘗試隔離每個路由器的行為：

• 嘗試使用 Nginx 入口指向的內部 Istio 平衡器的地址從集群中的容器調用 Knative 服務（即172.17.17.19和相應的Host header。如果這不起作用，則問題出在 Istio + Knative組合。

• 嘗試直接在 Nginx Ingress 后面運行一個 grpc 容器，並確保 Nginx 能夠通過 grpc 流量。

• 如果這兩者都有效，那么您的測試集群內流量與 Nginx 發送流量的方式之間存在一些差異。 我的猜測是轉發的流量缺少Host header，但我會先檢查上面概述的其他兩個調試步驟。