MicroK8s 容器无法启动。 所有 pod 都停留在“ContainerCreating”state 上。 Ubuntu 20.10 RP4

Question

I've been Googling and going through logs to try and solve this, but I can't seem to get microk8s to work on my Raspberry Pi, running Ubuntu 20.10

I snap install v1.15 sudo snap install microk8s --classic --channel=1.15/stable

I can confirm that microk8s.status returns that it is running. kubectl get nodes --namespace kube-system returns:

NAME         STATUS   ROLES    AGE   VERSION
mydnsname   Ready    <none>   22h   v1.15.11 

I have installed the following enabled: dashboard I had dashboard dns ingress - installed but the same issue was present.

kubectl get all --all-namespaces returns

NAMESPACE     NAME                                                  READY   STATUS              RESTARTS   AGE
kube-system   pod/heapster-v1.5.2-c4dcfd9f8-kl5g5                   0/4     ContainerCreating   0          68s
kube-system   pod/kubernetes-dashboard-c775bf88b-pmhz6              0/1     ContainerCreating   0          68s
kube-system   pod/monitoring-influxdb-grafana-v4-6f74479987-rd5ck   0/2     ContainerCreating   0          68s


NAMESPACE     NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
default       service/kubernetes             ClusterIP   10.152.183.1     <none>        443/TCP             18h
kube-system   service/heapster               ClusterIP   10.152.183.195   <none>        80/TCP              68s
kube-system   service/kubernetes-dashboard   ClusterIP   10.152.183.171   <none>        443/TCP             68s
kube-system   service/monitoring-grafana     ClusterIP   10.152.183.120   <none>        80/TCP              68s
kube-system   service/monitoring-influxdb    ClusterIP   10.152.183.175   <none>        8083/TCP,8086/TCP   68s


NAMESPACE     NAME                                             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/heapster-v1.5.2                  0/1     1            0           68s
kube-system   deployment.apps/kubernetes-dashboard             0/1     1            0           68s
kube-system   deployment.apps/monitoring-influxdb-grafana-v4   0/1     1            0           68s

NAMESPACE     NAME                                                        DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/heapster-v1.5.2-c4dcfd9f8                   1         1         0       68s
kube-system   replicaset.apps/kubernetes-dashboard-c775bf88b              1         1         0       68s
kube-system   replicaset.apps/monitoring-influxdb-grafana-v4-6f74479987   1         1         0       68s

kubectl describe pods kubernetes-dashboard-c775bf88b-pmhz6 --namespace kube-system returns

Warning  FailedCreatePodSandBox  2m26s (x3 over 2m54s)   kubelet, mydnsname  (combined from similar events): Failed create pod sandbox: rpc error: code = Unknown desc = failed to start sandbox container: failed to create containerd task: failed to mount rootfs component &{overlay overlay [workdir=/var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1386/work upperdir=/var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1386/fs lowerdir=/var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/2/fs]}: invalid argument: unknown
  Warning  MissingClusterDNS       2m14s (x13 over 4m58s)  kubelet, mydnsname  pod: "kubernetes-dashboard-c775bf88b-pmhz6_kube-system(777df7aa-d4de-40f0-8275-70b36d2e5029)". kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to "Default" policy.

I have checked and I am not using ZFS as my filesystem for my main partition, I'm using overlayfs df -Th

Filesystem     Type     Size  Used Avail Use% Mounted on
none           overlay   59G   21G   36G  37% /
tmpfs          tmpfs    1.9G     0  1.9G   0% /dev/shm
tmpfs          tmpfs    743M  1.2M  742M   1% /run
tmpfs          tmpfs    5.0M  4.0K  5.0M   1% /run/lock
tmpfs          tmpfs    4.0M     0  4.0M   0% /sys/fs/cgroup
tmpfs          tmpfs    372M  104K  372M   1% /run/user/1000

Therefore I'm not sure why containerd can't mount or has an issue, considering that containerd doesn't have any issues with overlayfs.

Have tried microk8s.reset and uninstalling and reinstalling, and tried a few different versions and still to no avail.

Any ideas or assistance is greatly appreciated as I'm keen to get a local kubernetes setup for local dev!

Thanks in advance!

Answer 1

Found the issue was with microk8s, specifically containerd not working with squashFS filesystem with overlay.

The reason for my Ubuntu install using SquashFS was that I was using Berryboot to support multiple OS installations.

I switched to Pinn, which supports ext4 filesystem, which works with containerd and reinstalled Ubuntu 20.04.

After installing microk8s, the node starts running and containers are all now running correctly.