KNative 和带有 nginx-ingress 的 gRPC

Question

我已经在我的 AWS EKS 集群中安装了 Knative/KServe。 一切正常，但最近我们决定为部署在那里的服务尝试 gRPC。 它是用 Istio 部署的，winth nginx ingress 在所有东西前面，ingress 指向 Istio ingress gateway：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
  name: computing-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /
        pathType: Prefix

如 Knative 文档https://github.com/meteatamel/knative-tutorial/blob/master/docs/grpc.md中所述，我通过添加 h2c 端口块更改了我的 InferenceService yaml：

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  annotations:
  finalizers:
  - inferenceservice.finalizers
  name: triton-test
  namespace: default
spec:
  predictor:
    model:
      args:
      - --model-control-mode=poll
      - --repository-poll-secs=5
      - --allow-grpc=true
      - --grpc-port=9000
      - --log-verbose=0
      env:
      - name: CUDA_VISIBLE_DEVICES
        value: "0"
      - name: S3_DATA_PATH
        value: s3://mymodeldata/
      - name: S3_PARAMS
        value: --region us-east-2 --no-sign-request
      image: XXXXXXX.dkr.ecr.us-east-2.amazonaws.com/ml:mytriton
      modelFormat:
        name: triton
      name: kserve-container
      ports:
      - containerPort: 9000
        name: h2c
        protocol: TCP
      protocolVersion: v2
      resources:
        limits:
          cpu: "2"
          memory: 8Gi
          nvidia.com/gpu: "1"
        requests:
          cpu: "2"
          memory: 8Gi
          nvidia.com/gpu: "1"
      storageUri: s3://mymodeldata/
      volumeMounts:
      - mountPath: /dev/shm
        name: dshm
    nodeSelector:
      DedicatedFor: GPU
    volumes:
    - emptyDir:
        medium: Memory
        sizeLimit: 2Gi
      name: dshm

由于 gRPC 注释是入口级的，我已经将主入口更改为具有更具体的路径：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
  name: computing-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /v1
        pathType: Prefix
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /v2
        pathType: Prefix

然后创建第二个入口

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
    nginx.ingress.kubernetes.io/grpc-backend: "true"
  name: computing-grpc-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80
        path: /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo
        pathType: ImplementationSpecific

但我无法让它以任何方式工作。 我尝试了一百万种不同的配置，但出现 404 或 502 错误。 我的 istio 入口服务是：

apiVersion: v1
kind: Service
metadata:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /healthz/ready
    alb.ingress.kubernetes.io/healthcheck-port: "31619"
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: unknown
    istio: ingressgateway
    istio.io/rev: default
    operator.istio.io/component: IngressGateways
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
spec:
  clusterIP: 172.17.17.19
  clusterIPs:
  - 172.17.17.19
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: status-port
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: http2
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  sessionAffinity: None
  type: ClusterIP

有什么办法让它工作吗？ 我不确定我应该添加哪些附加信息。 谢谢！

Answer 1

由于您将两个不同的 HTTP 路由器链接在一起，您可能想尝试隔离每个路由器的行为：

• 尝试使用 Nginx 入口指向的内部 Istio 平衡器的地址从集群中的容器调用 Knative 服务（即172.17.17.19和相应的Host header。如果这不起作用，则问题出在 Istio + Knative组合。

• 尝试直接在 Nginx Ingress 后面运行一个 grpc 容器，并确保 Nginx 能够通过 grpc 流量。

• 如果这两者都有效，那么您的测试集群内流量与 Nginx 发送流量的方式之间存在一些差异。 我的猜测是转发的流量缺少Host header，但我会先检查上面概述的其他两个调试步骤。