豆荚死了。 容器在K8s 1.13中启动

Question

有没有人遇到过这样一个场景，即该Pod从一个节点（即节点A）中逐出，然后被立即调度到另一个节点（即节点B）上。 但是节点A的kubelet一直抱怨无法删除pod的容器。 容器已经装好，没有被杀死。

Aug 30 20:29:36 staging-node-4 kubelet[2173]: I0830 20:29:36.358238    2173 kubelet_pods.go:1073] Killing unwanted pod "thanos-compactor-0"
Aug 30 20:29:36 staging-node-4 kubelet[2173]: I0830 20:29:36.362581    2173 kuberuntime_container.go:559] Killing container "docker://b22287cd406c3fe9eff4ff2df1792c6f84b5b92d001359f05ea73f8788715609" with 30 second grace period
Aug 30 20:29:36 staging-node-4 kubelet[2173]: E0830 20:29:36.363416    2173 kuberuntime_container.go:71] Can't make a ref to pod "thanos-compactor-0_thanos(23096418-c7b2-11e9-9be7-005056bc2883)", container thanos: selfLink was empty, can't make reference
Aug 30 20:29:38 staging-node-4 kubelet[2173]: I0830 20:29:38.362191    2173 kubelet_pods.go:1073] Killing unwanted pod "thanos-compactor-0"
Aug 30 20:29:38 staging-node-4 kubelet[2173]: I0830 20:29:38.366884    2173 kuberuntime_container.go:559] Killing container "docker://b22287cd406c3fe9eff4ff2df1792c6f84b5b92d001359f05ea73f8788715609" with 30 second grace period
Aug 30 20:29:38 staging-node-4 kubelet[2173]: E0830 20:29:38.367901    2173 kuberuntime_container.go:71] Can't make a ref to pod "thanos-compactor-0_thanos(23096418-c7b2-11e9-9be7-005056bc2883)", container thanos: selfLink was empty, can't make reference
Aug 30 20:29:38 staging-node-4 kubelet[2173]: I0830 20:29:38.997510    2173 setters.go:72] Using node IP: "172.17.21.146"
Aug 30 20:29:40 staging-node-4 kubelet[2173]: I0830 20:29:40.364388    2173 kubelet_pods.go:1073] Killing unwanted pod "thanos-compactor-0"
Aug 30 20:29:40 staging-node-4 kubelet[2173]: I0830 20:29:40.368226    2173 kuberuntime_container.go:559] Killing container "docker://b22287cd406c3fe9eff4ff2df1792c6f84b5b92d001359f05ea73f8788715609" with 30 second grace period
Aug 30 20:29:40 staging-node-4 kubelet[2173]: E0830 20:29:40.369235    2173 kuberuntime_container.go:71] Can't make a ref to pod "thanos-compactor-0_thanos(23096418-c7b2-11e9-9be7-005056bc2883)", container thanos: selfLink was empty, can't make reference

b22287cd是未删除的Docker容器ID。

Answer 1

几周前我遇到了这个问题。 荚被从节点上逐出，有时被“卡住”在终止状态。

通过运行kubectl -n <NAMESPACE> get pod检查您的pod状态

使用kubectl delete pod <PODNAME> --grace-period=0 --force --namespace <NAMESPACE>解决了该问题。