Kubernetes unusable when disconnected from internet (Windows)

Question

I'm using Docker for Windows with Kubernetes. If I'm disconnected from the internet and restart my computer or restart Kubernetes, then it gets stuck in a perpetual kubernetes is starting... mode. I can run kubectl proxy but anything else fails.

eg kubectl get pod gives me Unable to connect to the server: EOF

Edit: Solution

• Uncheck the Automatically check for updates box in the Kubernetes General Settings fixed it for me.
• (optional) Change your deployments to use imagePullPolicy: IfNotPresent . I did this for my kubernetes-dashboard deployment.

Oddly, the kubernetes status would still get stuck in Kubernetes is starting... even though I was able to interact with it via kubectl

Following @aurleius's answer, I tried patching my compose and compose-api deployments but those settings were lost whenever I reset via the docker right click menu. I wrote a powershell script to patch the deployment so I'm putting it here just in case.

# Patch compose
kubectl patch deployment compose -n docker -p "{ \`"spec\`": { \`"template\`": { \`"spec\`": { \`"containers\`": [{ \`"name\`": \`"compose\`", \`"imagePullPolicy\`": \`"IfNotPresent\`" }] } } } }"

# Patch compose-api
kubectl patch deployment compose-api -n docker -p "{ \`"spec\`": { \`"template\`": { \`"spec\`": { \`"containers\`": [{ \`"name\`": \`"compose\`", \`"imagePullPolicy\`": \`"IfNotPresent\`" }] } } } }"

Answer 1

I have tested your scenario on Mac and Windows and a short answer to this is that by default you require Internet connection to run Kubernetes cluster correctly.

The reason for that is specified in the documentation :

An Internet connection is required. Images required to run the Kubernetes server are downloaded and instantiated as containers, and the Program Files\\Docker\\Docker\\Resources\\bin\\kubectl.exe` command is installed.

What the documentation is not specifying is that the images which are used to run Kubernetes on Docker are possibly instantly checking for updates and new images for docker pods.

On Windows you can see that when you turn off the internet, close Docker and then run it again you can see that:

PS C:\Users\Administrator> kubectl get pods --all-namespaces
NAMESPACE     NAME                                         READY   STATUS             RESTARTS   AGE
docker        compose-7447646cf5-hzdbl                     0/1     CrashLoopBackOff   0          21m
docker        compose-api-6fbc44c575-b5b47                 0/1     CrashLoopBackOff   1          21m
kube-system   etcd-docker-for-desktop                      1/1     Running            1          20m
kube-system   kube-apiserver-docker-for-desktop            1/1     Running            1          20m
kube-system   kube-controller-manager-docker-for-desktop   1/1     Running            1          20m
kube-system   kube-dns-86f4d74b45-chzdc                    3/3     Running            3          21m
kube-system   kube-proxy-xsksv                             1/1     Running            1          21m
kube-system   kube-scheduler-docker-for-desktop            1/1     Running            1          20m


> PS C:\Users\Administrator> kubectl get pods -n kube-system Unable to
> connect to the server: EOF

Machines go to CrashLoopBackOff or ImagePullBackOff so Kubernetes Cluster is not running because it can't download new images according to it's policies. I have found how to prevent this error:

PS C:\\Users\\Administrator> kubectl get deployments --all-namespaces NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE docker compose 1 1 1 1 33m docker compose-api 1 1 1 1 33m kube-system kube-dns 1 1 1 1 33m

You can see the deployments and now we can change the Image Pull Policy to IfNotPresent. Note that we have to do it with both deployments in docker namespace. Edit it: PS C:\\Users\\Administrator> kubectl edit deployment compose -n docker

spec:
      containers:
      - args:
        - --kubeconfig
        - ""
        - --reconciliation-interval
        - 30s
        image: docker/kube-compose-controller:v0.3.9
        imagePullPolicy: Alwaysspec:
      containers:
      - args:
        - --kubeconfig
        - ""
        - --reconciliation-interval
        - 30s
        image: docker/kube-compose-controller:v0.3.9
        imagePullPolicy: IfNotPresent    

The difference between Mac and Windows is that Mac shows the error after some time while Windows ends in a loop. Hope this helps.

Update: From what I've seen there are several scenarios. Interestingly the update checkbox had no impact on these events: 1) editing deployments and offline restart (laptop restart) does not overwrite the imagePullPolicy 2) editing deployments and online laptop restart does not overwrite the imagePullPolicy 3) if by restart you understood the Cluster restart option in the Docker menu, then yes it overwrites all the deployment files. I looked for those yaml files but they are nowhere to be found in Windows file system, also I am not sure if that would work since it would change the checksum of those files and Docker could not take it. Other option is that it might be just impossible since the docker on Windows:

Images required to run the Kubernetes server are downloaded and instantiated as containers