Kubernetes: Handle connections with multiple LoadBalancer in cluster via traefik ingress controller

Question

It might be hard to explain so sorry if ı can not explain correctly.

In our k8s cluster we have two OpenStack-Load Balancer because we would like to expose our application through ingress which has to be internet facing. In same cluster we also deployed pgadmin4 which has to be intranet facing.(only reachable from internal network.)

So in front of these OpenStack-LB, we have also f5 Load Balancer which handle https connection,ssl .. and also logic to expose via intranet or internet.

MyApp is internet facing and needs to reachable with host.internet.net

PgAdmin4 is intranet and needs to reachable via host.intranet.net/pgadmin4

So the issue is, when I try to expose my application through ingress using host.internet.net it won't works and ı received below error cause probably it can not able to communicate with correct openStack-LB. When ı tried to expose via openStack-lb IP everything works properly.

{"level":"error","msg":"Service not found for dev/oneapihub-ui-dev","time":"2020-03-26T05:20:05Z"} {"level":"error","msg":"endpoints not found for dev/oneapihub-ui-dev","time":"2020-03-26T05:20:05Z"}

And the question is , how can I handle this issue via ingress controller? Should I intall another traefik ingress controller?

capel0068340585:~ semural$ kubectl get ingress -n ingress
NAME                        HOSTS   ADDRESS   PORTS   AGE
ingress-traefik-dashboard   *                 80      21d

---

kubectl get tenantSpec -o yaml

    loadBalancers:
    - ip: <IP1>
      name: LBaaS2
      ports:
      - extPort: 80
        name: "80"
        nodePort: 30001
    - ip: <IP2>
      name: LBaaS1
      ports:
      - extPort: 80
        name: "80"
        nodePort: 30000

---

NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/oneapihub-ui-dev        ClusterIP   10.254.173.130   <none>        80/TCP     15m

NAME                        ENDPOINTS           AGE
endpoints/oneapihub-ui-dev        10.6.24.136:3000    15m

---

ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: traefik
  hosts:
    - host:  host.internet.net -> example
      paths: [/]
  tls: []

---

ingress:
  enabled: ingress
  annotations: 
    kubernetes.io/ingress.class: traefik
  hosts:
    - host: host.intranet.net
      paths:
      - /pgadmin4

Answer 1

You error state "Service not found for dev/oneapihub-ui-dev" , which means traefik is trying to connect to a Service in the dev namespace called "oneapihub-ui-dev" which it cannot find.

You need to make sure that both the Service exists and that it has endpoints. You can check if the Service exists with kubectl -n dev get service oneapihub-ui-dev . If it exists, check if it has endpoints with kubectl -n dev get ep oneapihub-ui-dev .

EDIT: If the Service exists and has Endpoints, than you may want to look into the RBAC permissions of traefik to see if it has enough permissions to look in the dev namespace and if you do not deploy any NetworkPolicies on the dev namespace that prevent the ingress namespace from connecting.

Answer 2

I solved this issue using via using labelSelector for traefik.. The the services that I'd expose only for internal networking has a label such as traffic-type=internal.. You could also provide a namespace for RBAC permissions.

kubernetes:
  namespaces:
   - default
   - database
   - monitoring
   - logging
   - ingress
  labelSelector: "traffic-type=internal"