kubernetes nginx ingress GRPC for one path

Question

I am trying to setup an nginx kubernetes ingress . I am able to serve http and websockets content on different routes at the moment.

However I am not able to add GRPC routes on the same host. Adding this annotation nginx.ingress.kubernetes.io/backend-protocol: "GRPC" breaks the existing routes.

My java GRPC client exits with Caused by: io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception: First received frame was not SETTINGS. Hex dump for first 5 bytes: 485454502f Caused by: io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception: First received frame was not SETTINGS. Hex dump for first 5 bytes: 485454502f According to https://github.com/grpc/grpc-java/issues/2905 this means the request is seen as HTTP

Is there a way to have http/websocket/grpc routes on the same host using the nginx kubernetes ingress? Alternatively, is there another ingress with which this would work?

Answer 1

As you want the annotation nginx.ingress.kubernetes.io/backend-protocol: "GRPC" to apply only on certain routes of your host, you could declare two Ingress definitions. The first one for all HTTP routes, the second one for GRPC routes.

The Nginx Ingress Controller will pick all the Ingress definitions (with the expected IngressClass ) and will use them to compose the nginx.conf . This behaviour is perfect to have the possibility of having paths which requires different tunings in the annotations, like rewrite targets or, in your case, different backend protocols.

In particular, from the Nginx Controller documentation:

Multiple Ingresses can define different annotations. These definitions are not shared between Ingresses.

You can check all the steps which are used to build the nginx.conf in the docs: https://kubernetes.github.io/ingress-nginx/how-it-works/#building-the-nginx-model