Can't access minikube service using NodePort from host on Mac

Question

I'm trying to deploy a single web application to Minikube on my Mac, and then access it in the browser. I'm trying to use the simplest of setups, but it's not working, I just get a "connection refused" error and I can't figure out why.

This is what I'm trying:

$ minikube start --insecure-registry=docker.example.com:5000
😄  minikube v1.12.3 on Darwin 10.14.6
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing docker container for "minikube" ...
🐳  Preparing Kubernetes v1.18.3 on Docker 19.03.8 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube"

$ eval $(minikube -p minikube docker-env)

$ docker build -t web-test .
Sending build context to Docker daemon  16.66MB
Step 1/3 : FROM docker.example.com/library/openjdk:11-jdk-slim
11-jdk-slim: Pulling from library/openjdk
bf5952930446: Pull complete 
092c9b8e633f: Pull complete 
0b793152b850: Pull complete 
7900923f09cb: Pull complete 
Digest: sha256:b5d8f95b23481a9d9d7e73c108368de74abb9833c3fae80e6bdfa750663d1b97
Status: Downloaded newer image for docker.example.com/library/openjdk:11-jdk-slim
 ---> de8b1b4806af
Step 2/3 : COPY target/web-test-0.0.1-SNAPSHOT.jar app.jar
 ---> 6838e3db240a
Step 3/3 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","app.jar"]
 ---> Running in 550bf762bf2d
Removing intermediate container 550bf762bf2d
 ---> ce1468d1ff10
Successfully built ce1468d1ff10
Successfully tagged web-test:latest

$ kubectl apply -f web-test-service.yaml 
service/web-test unchanged

$ kubectl apply -f web-test-deployment.yaml 
deployment.apps/web-test configured

$ kubectl get po -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
web-test-6bb45ffc54-8mxbc   1/1     Running   0          16m   172.18.0.2   minikube   <none>           <none>

$ kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          16m
web-test     NodePort    10.102.19.201   <none>        8080:31317/TCP   16m

$ minikube ip
127.0.0.1

$ curl http://127.0.0.1:31317
curl: (7) Failed to connect to 127.0.0.1 port 31317: Connection refused


$ kubectl logs web-test-6bb45ffc54-8mxbc

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.3.RELEASE)

2020-08-26 14:45:32.692  INFO 1 --- [           main] com.example.web.WebTestApplication           : Starting WebTestApplication v0.0.1-SNAPSHOT on web-test-6bb45ffc54-8mxbc with PID 1 (/app.jar started by root in /)
2020-08-26 14:45:32.695  INFO 1 --- [           main] com.example.web.WebTestApplication           : No active profile set, falling back to default profiles: default
2020-08-26 14:45:34.041  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer      : Tomcat initialized with port(s): 8080 (http)
2020-08-26 14:45:34.053  INFO 1 --- [           main] o.apache.catalina.core.StandardService       : Starting service [Tomcat]
2020-08-26 14:45:34.053  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine      : Starting Servlet engine: [Apache Tomcat/9.0.37]
2020-08-26 14:45:34.135  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]           : Initializing Spring embedded WebApplicationContext
2020-08-26 14:45:34.135  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext     : Root WebApplicationContext: initialization completed in 1355 ms
2020-08-26 14:45:34.587  INFO 1 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor      : Initializing ExecutorService 'applicationTaskExecutor'
2020-08-26 14:45:34.797  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer      : Tomcat started on port(s): 8080 (http) with context path ''
2020-08-26 14:45:34.810  INFO 1 --- [           main] com.example.web.WebTestApplication           : Started WebTestApplication in 2.808 seconds (JVM running for 3.426)


$ minikube ssh
docker@minikube:~$ curl 10.102.19.201:8080
Up and Running
docker@minikube:~$

As you can see, the web app is up and running, and I can access it from inside the cluster by doing a minikube ssh , but from outside the cluster, it won't connect. These are my service and deployment manifests:

web-test-service.yaml:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: web-test
  name: web-test
spec:
  type: NodePort
  ports:
  - nodePort: 31317
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: web-test

web-test-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web-test
  name: web-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-test
  strategy: {}
  template:
    metadata:
      labels:
        app: web-test
    spec:
      containers:
      - image: web-test
        imagePullPolicy: Never
        name: web-test
        ports:
        - containerPort: 8080
      restartPolicy: Always
status: {}

Anyone have any idea what I'm doing wrong? Or perhaps how I could try to diagnose the issue further? I have allow tried deploying an ingress, but that doesn't work either.

Answer 1

You are mostly facing this issue when you use minikube ip which returns 127.0.0.1 . It should work if you use internal ip from kubectl get node -o wide instead of 127.0.0.1 .

A much easier approach from the official reference docs is you can get the url using minikube service web-test --url and use it in browser or if you use minikube service web-test it will open the url in browser directly.

Your deployment yamls and everything else looks good and hopefully should not have any issue when deploying to a remote cluster.

Answer 2

It seems that is related to the default docker driver used when you start the minikube. To avoid these problems you can force a specific driver (eg "virtualbox"). To do so, follow the next steps:

Remove old minikube with:

minikube delete

Start minikube with virtualbox driver:

minikube start --memory=4096 --driver=virtualbox

Run minikube ip . You'll see an output like 192.168.99.100 . Then, create again the Pods and the service and it should work properly. I've found this info in this issue: https://github.com/kubernetes/minikube/issues/7344#issuecomment-703225254

Answer 3

You can export an Service from minikube with minikube service web-test

https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-service

Edit:

If you have a deployment, you can export that deployment with the following kubectl command.

minikube kubectl -- expose deployment your-deployment --port 80 --type=LoadBalancer

Answer 4

Just in case you have not already stumbled across a broader concept for accessing a nodeport service that applies in general vs proprietary minikube constructs:

$ k get service -A NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP
default nginx LoadBalancer 10.43.228.207 172.22.0.240 80:30467/TCP 11h

$ kubectl port-forward --address 0.0.0.0 service/nginx 8082:80

Then from a different host on my network, I do: curl [the host running minikube]:8082

Forwarding from 0.0.0.0:8082 -> 80 Handling connection for 8082

Then you can connect from a different host as well.

Answer 5

docker-desktop UI for Mac and Windows provides an easier alternative compared to minikube, which you could simply activate the Kubernetes feature on your docker-desktop UI :

once it is setup you can right click on the docker desktop icon > Kubernetes

To verify now that your deployement/service works properly:

kubectl apply -f /file.yaml

Answer 6

One checkpoint we should keep in mind for ports.

targetPort: 80

Belongs to the port which we have exposed in our(Dockerfile or Docker-compose file). If the port is mismatched you won't be able to access it.

Answer 7

The answer is never use Minikube. It does not allow you to use Nodeport connections. You will always get ECONNREFUSED with minikube no matter what. Just use the docker desktop context, kill minikube, and then re-apply your services. Minikube is only there to further confuse people who are learning Kubernetes.