coredns pod is not running in kubernetes

Question

I install kubernetes 1000 times but now it does not work. I install kubectl kubeadm kubelet then

 sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=185.73.114.92
 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

but I see coredns is in pending state

kubectl get pods --all-namespaces
NAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE
kube-system   coredns-5644d7b6d9-492q4          0/1     Pending   0          13m
kube-system   coredns-5644d7b6d9-cvwjg          0/1     Pending   0          13m
kube-system   etcd-amghezi                      1/1     Running   0          12m
kube-system   kube-apiserver-amghezi            1/1     Running   0          12m
kube-system   kube-controller-manager-amghezi   1/1     Running   0          12m
kube-system   kube-flannel-ds-amd64-fkxnf       1/1     Running   0          12m
kube-system   kube-proxy-pspw2                  1/1     Running   0          13m
kube-system   kube-scheduler-amghezi            1/1     Running   0          12m

and then I get describe of coredns

kubectl describe pods coredns-5644d7b6d9-492q4 -n kube-system
Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.
  Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.

I taint the node by

kubectl taint nodes amghezi node-role.kubernetes.io/master-

It did not work I see at

journalctl -xe
message:docker: network plugin is not ready: cni config uninitialized

service docker status
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
   Active: active (running) since Sun 2019-09-22 17:29:45 CEST; 34min ago
     Docs: https://docs.docker.com
 Main PID: 987 (dockerd)
    Tasks: 20
   CGroup: /system.slice/docker.service
           └─987 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Sep 22 17:29:45 ubuntu systemd[1]: Started Docker Application Container Engine.
Sep 22 17:29:45 ubuntu dockerd[987]: time="2019-09-22T17:29:45.728818467+02:00" level=info msg="API listen on /var/run/docker.sock"
Sep 22 17:29:45 ubuntu dockerd[987]: time="2019-09-22T17:29:45.757401709+02:00" level=warning msg="failed to retrieve runc version: unknown output format: runc version spec: 1.0.1-dev\n"
Sep 22 17:29:45 ubuntu dockerd[987]: time="2019-09-22T17:29:45.786776798+02:00" level=warning msg="failed to retrieve runc version: unknown output format: runc version spec: 1.0.1-dev\n"
Sep 22 17:29:46 ubuntu dockerd[987]: time="2019-09-22T17:29:46.296798944+02:00" level=warning msg="failed to retrieve runc version: unknown output format: runc version spec: 1.0.1-dev\n"
Sep 22 17:29:46 ubuntu dockerd[987]: time="2019-09-22T17:29:46.364459982+02:00" level=warning msg="failed to retrieve runc version: unknown output format: runc version spec: 1.0.1-dev\n"
Sep 22 17:30:06 ubuntu dockerd[987]: time="2019-09-22T17:30:06.996299645+02:00" level=warning msg="failed to retrieve runc version: unknown output format: runc version spec: 1.0.1-dev\n"
Sep 22 17:30:41 ubuntu dockerd[987]: time="2019-09-22T17:30:41.633452599+02:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 22 17:30:41 ubuntu dockerd[987]: time="2019-09-22T17:30:41.633831003+02:00" level=warning msg="d72e19bd0e929513a1c9092ec487e5dc3f3e009bdaa4d33668b610e86cdadf9e cleanup: failed to unmount IPC: umount /var/lib/docker/containers/d72e19bd0e929513a1c9092ec487e5dc3f3e009bdaa4d33668b610e86cdadf9e/mounts/shm, flags: 0x2
Sep 22 17:30:41 ubuntu dockerd[987]: time="2019-09-22T17:30:41.903058543+02:00" level=warning msg="Your kernel does not support swap limit capabilities,or the cgroup is not mounted. Memory limited without swap."

and let us see kubelet status

Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

Answer 1

I assume per given data outputs that the problem comes fromKubelet node agent, since kubelet primarily demands on CNI network plugin installation. In order to automatically configure networking features for the Pods, kubelet starts each time CNI addon in prior Pod creation to set up pod's network interface as well. Furthermore, CoreDNS discovery service relies on overlay container network to be reachable for all cluster nodes.

Although you've used Flannel CNI provider, flannel Pod is up and running, since kubelet can't create container interface for the particular CoreDNS Pods with lack of CNI configuration, I would recommend to reset kubeadm cluster with purging redundant component folder structure:

$ sudo kubeadm reset

$ sudo systemctl stop docker && sudo systemctl stop kubelet

$ sudo rm -rf /etc/kubernetes/

$ sudo rm -rf.kube/

$ sudo rm -rf /var/lib/kubelet/

$ sudo rm -rf /var/lib/cni/

$ sudo rm -rf /etc/cni/

$ sudo rm -rf /var/lib/etcd/

Bootstrap K8s cluster via kubeadm :

$ sudo systemctl start docker && sudo systemctl start kubelet

$ sudo kubeadm init...

Further remove node-role.kubernetes.io/master taint and apply Flannel addon:

$ kubectl taint nodes --all node-role.kubernetes.io/master-

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

You might find also some useful information about kubeadm troubleshooting guide steps in the official K8s documentation.