如何使用 nginx-ingress controller 公开 TCP 的多个服务？

Question

I have multiple deployments running of RDP application and they all are exposed with ClusterIP service. I have nginx-ingress controller in my k8s cluster and to allow tcp I have added --tcp-services-configmap flag in nginx-ingress controller deployment and also created a configmap for the same that is shown below

apiVersion: v1 
kind: ConfigMap
metadata:
name: tcp-services
namespace: ingress-nginx
data:
  3389: “demo/rdp-service1:3389”

This will expose “rdp-service1” service. And I have 10 more such services which needed to be exposed on the same port number but if I add more service in the same configmap like this

...
data
  3389: “demo/rdp-service1:3389”
  3389: “demo/rdp-service2:3389”

Then it will remove the previous service data and since here I have also deployed external-dns in k8s, so all the records created by ingress using host: ... will starts pointing to the deployment attached with the newly added service in configmap.

Now my final requirement is as soon as I append the rule for a newly created deployment(RDP application) in the ingress then it starts allowing the TCP connection for that, so is there any way to achieve this. Or is there any other Ingress controller available that can solve such type of use case and can also easily be integrated with external-dns?

Note:- I am using AWS EKS Cluster and Route53 with external-dns.

Answer 1

Posting this answer as a community wiki to explain some of the topics in the question as well as hopefully point to the solution.

Feel free to expand/edit it.

---

NGINX Ingress main responsibility is to forward the HTTP / HTTPS traffic. With the addition of the tcp-services / udp-services it can also forward the TCP / UDP traffic to their respective endpoints:

• Kubernetes.github.io: Ingress nginx: User guide: Exposing tcp udp services

The main issue is that the Host based routing for Ingress resource in Kubernetes is targeting specifically HTTP / HTTPS traffic and not TCP ( RDP ).

You could achieve a following scenario:

• Ingress controller :
  • 3389 - RDP Deployment #1
  • 3390 - RDP Deployment #2
  • 3391 - RDP Deployment #3

Where there would be no Host based routing. It would be more like port-forwarding.

A side note! This setup would also depend on the ability of the LoadBalancer to allocate ports (which could be limited due to cloud provider specification)

---

As for possible solution which could be not so straight-forward I would take a look on following resources:

• Stackoverflow.com: Questions: Nxing TCP forwarding based on hostname
• Doc.traefik.io: Traefik: Routing: Routers: Configuring TCP routers
• Github.com: Bolkedebruin: Rdpgw

I'd also check following links:

• Aws.amazon.con: Quickstart: Architecture: Rd gateway - AWS specific

• Docs.konghq.com: Kubernetes ingress controller: 1.2.X: Guides: Using tcpingress

• Haproxy:

  • Haproxy.com: Documentation: Aloha: 12-0: Deployment guides: Remote desktop: RDP gateway
  • Haproxy.com: Documentation: Aloha: 10-5: Deployment guides: Remote desktop
  • Haproxy.com: Blog: Microsoft remote desktop services rds load balancing and protection

Answer 2

Actually, I really don't know why you are using that configmap.

In my knowledge, nginx-ingress-controller is routing traffic coming in the same port and routing based on host. So if you want to expose your applications on the same port, try using this:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: {{ .Chart.Name }}-ingress
  namespace: your-namespace
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: your-hostname
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          serviceName: {{ .Chart.Name }}-service
          servicePort: {{ .Values.service.nodeport.port }}

Looking in your requirement, I feel that you need a LoadBalancer rather than Ingress