简体繁体 English

Kubernetes pod 更新失败

[英]Kubernetes pod failed to update

原文 2021-11-23 09:45:46 7 1 kubernetes

We have a Gitlab CI/CD to deploy pod via Kubernetes.我们有一个 Gitlab CI/CD 通过 Kubernetes 部署 pod。 However, the updated pod is always pending and the deleted pod is always stuck at terminating.但是，更新的 pod 始终处于挂起状态，而删除的 pod 始终停留在终止状态。

The controller and scheduler are both okay. controller 和调度器都可以。

If I described the pending pod, it shows it is scheduled but nothing else.如果我描述了待处理的 pod，它显示它已被安排，但没有别的。

This is the pending pod's logs:这是待处理 pod 的日志：

$ kubectl logs -f robo-apis-dev-7b79ccf74b-nr9q2 -n xxx -f Error from server (BadRequest): container "robo-apis-dev" in pod "robo-apis-dev-7b79ccf74b-nr9q2" is waiting to start: ContainerCreating $ kubectl logs -f robo-apis-dev-7b79ccf74b-nr9q2 -n xxx -f 来自服务器的错误（BadRequest）：pod“robo-apis-dev-7b79ccf74b-nr9q2”中的容器“robo-apis-dev”正在等待开始：容器创建

What could be the issue?可能是什么问题？ Our Kubernetes cluster never had this issue before.我们的 Kubernetes 集群以前从未遇到过此问题。

1 个解决方案

Is your pod is supposed to stay up?你的吊舱应该熬夜吗？ What happen if you summon it on docker for example?例如，如果你在 docker 上召唤它会发生什么？ Do you have an healthcheck on the deploy/pod definition?您是否对部署/pod 定义进行了健康检查？ Seems like it time out seeing the Pending for 19min and terminating at 26min.似乎看到 Pending 19 分钟并在 26 分钟终止。

Okay, it turns out we used to have an NFS server as PVC.好的，原来我们曾经有一个 NFS 服务器作为 PVC。 But we have moved to AWS EKS recently, thus cleaning the NFS servers.但我们最近已迁移到 AWS EKS，从而清理了 NFS 服务器。 Maybe there are some resources from nodes that are still on the NFS server.也许有一些来自仍然在 NFS 服务器上的节点的资源。 Once we temporarily roll back the NFS server, the pods start to move to RUNNING state.一旦我们暂时回滚 NFS 服务器，pod 就会开始移动到 RUNNING state。