exposing CockroachDB on Kubernetes to public IP

Question

I have a CockroachDB instance running in a Kubernetes cluster on Google Kubernetes Engine. I am trying to expose port 26257 so I can connect to it from my local machine.

As stated in this answer , port forwarding to the pod will not work.

I have an nginx-ingress controller which is used to map from my domain name paths to services, so I tried to use that:

I changed my db-cockroachdb-public service from ClusterIP to NodePort:

type: NodePort

I added these lines to my nginx-controller YAML:

-name: postgresql

nodePort: 30472

port: 26257

protocol: TCP

targetPort: 26257

and these lines to my ingress YAML:

- host: db.mydomain.com
  http:
    paths:
    - path: /
      backend:
        serviceName: db-cockroachdb-public
        servicePort: 26257

However, I'm unable to connect to the database - connection gets refused. I also tried to disable SSL redirects in the nginx controller, but it still doesn't work.

I also tried a ConfigMap but it didn't do anything:

https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/exposing-tcp-udp-services.md

Answer 1

There are a few ways to fix this. Most are related to changing your ingress configuration or how you're connecting to the service, which I'm not going to go into. Another option is to make port forwarding work to eliminate the need for the ingress machinery.

You can make port forwarding work by modifying the CockroachDB config file slightly. Change the name of the --host flag in the invocation of the Cockroach binary to be --advertise-host instead. That way, the process will listen on localhost in addition to on its hostname, which will make port forwarding work.

edit: To follow up on this, I've switched the default configuration in the CockroachDB repo to use --advertise-host instead of --host , so port forwarding works by default now.

Answer 2

I don't know if it technically should work to proxy a CockroachDB through a nginx instance, but your setup fails for another reason. When specifying a servicePort in the rules section, you tell k8s which port is exposed to the service. The mapping itself happens by default to port 80/443, not your desired port. So you should try just to ask port 80 in your case.