执行和初始化 kubeadm 时出错

Question

初始化kubeadm时出现以下错误。 在执行kubadm init之前，我也尝试过命令kubeadm reset 。 Kubelet 也在运行，我使用的相同命令是systemctl enable kubelet && systemctl start kubelet 。 以下是执行 kubeadm init 后的日志

 [kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters. [init] Using Kubernetes version: v1.8.2 [init] Using Authorization modes: [Node RBAC] [preflight] Running pre-flight checks [preflight] WARNING: Connection to "https://192.168.78.48:6443" uses proxy "http://user:pwd@192.168.78.15:3128/". If that is not intended, adjust your proxy settings [preflight] WARNING: Running with swap on is not supported. Please disable swap or set kubelet's --fail-swap-on flag to false. [preflight] Starting the kubelet service [kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0) [certificates] Generated ca certificate and key. [certificates] Generated apiserver certificate and key. [certificates] apiserver serving cert is signed for DNS names [steller.india.com kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.140.48] [certificates] Generated apiserver-kubelet-client certificate and key. [certificates] Generated sa key and public key. [certificates] Generated front-proxy-ca certificate and key. [certificates] Generated front-proxy-client certificate and key. [certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki" [kubeconfig] Wrote KubeConfig file to disk: "admin.conf" [kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf" [kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf" [kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf" [controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml" [controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml" [controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml" [etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml" [init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests" [init] This often takes around a minute; or longer if the control plane images have to be pulled. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp [::1]:10255: getsockopt: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp [::1]:10255: getsockopt: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp [::1]:10255: getsockopt: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz/syncloop' failed with error: Get http://localhost:10255/healthz/syncloop: dial tcp [::1]:10255: getsockopt: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz/syncloop' failed with error: Get http://localhost:10255/healthz/syncloop: dial tcp [::1]:10255: getsockopt: connection refused.

以下是journalctl -u kubelet输出

 -- Logs begin at Thu 2017-11-02 16:20:50 IST, end at Fri 2017-11-03 17:11:12 IST. -- Nov 03 16:36:48 steller.india.com systemd[1]: Started kubelet: The Kubernetes Node Agent. Nov 03 16:36:48 steller.india.com systemd[1]: Starting kubelet: The Kubernetes Node Agent... Nov 03 16:36:48 steller.india.com kubelet[52511]: I1103 16:36:48.998467 52511 feature_gate.go:156] feature gates: map[] Nov 03 16:36:48 steller.india.com kubelet[52511]: I1103 16:36:48.998532 52511 controller.go:114] kubelet config controller: starting controller Nov 03 16:36:48 steller.india.com kubelet[52511]: I1103 16:36:48.998536 52511 controller.go:118] kubelet config controller: validating combination of defaults and flag Nov 03 16:36:49 steller.india.com kubelet[52511]: I1103 16:36:49.837248 52511 client.go:75] Connecting to docker on unix:///var/run/docker.sock Nov 03 16:36:49 steller.india.com kubelet[52511]: I1103 16:36:49.837282 52511 client.go:95] Start docker client with request timeout=2m0s Nov 03 16:36:49 steller.india.com kubelet[52511]: W1103 16:36:49.839719 52511 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d Nov 03 16:36:49 steller.india.com systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE Nov 03 16:36:49 steller.india.com kubelet[52511]: I1103 16:36:49.846959 52511 feature_gate.go:156] feature gates: map[] Nov 03 16:36:49 steller.india.com kubelet[52511]: W1103 16:36:49.847216 52511 server.go:289] --cloud-provider=auto-detect is deprecated. The desired cloud provider sho Nov 03 16:36:49 steller.india.com kubelet[52511]: error: failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such Nov 03 16:36:49 steller.india.com systemd[1]: Unit kubelet.service entered failed state. Nov 03 16:36:49 steller.india.com systemd[1]: kubelet.service failed. Nov 03 16:37:00 steller.india.com systemd[1]: kubelet.service holdoff time over, scheduling restart. Nov 03 16:37:00 steller.india.com systemd[1]: Started kubelet: The Kubernetes Node Agent. Nov 03 16:37:00 steller.india.com systemd[1]: Starting kubelet: The Kubernetes Node Agent... Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.134702 52975 feature_gate.go:156] feature gates: map[] Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.134763 52975 controller.go:114] kubelet config controller: starting controller Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.134767 52975 controller.go:118] kubelet config controller: validating combination of defaults and flag Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.141273 52975 client.go:75] Connecting to docker on unix:///var/run/docker.sock Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.141364 52975 client.go:95] Start docker client with request timeout=2m0s Nov 03 16:37:00 steller.india.com kubelet[52975]: W1103 16:37:00.143023 52975 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.149537 52975 feature_gate.go:156] feature gates: map[] Nov 03 16:37:00 steller.india.com kubelet[52975]: W1103 16:37:00.149780 52975 server.go:289] --cloud-provider=auto-detect is deprecated. The desired cloud provider sho Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.179873 52975 certificate_manager.go:361] Requesting new certificate. Nov 03 16:37:00 steller.india.com kubelet[52975]: E1103 16:37:00.180392 52975 certificate_manager.go:284] Failed while requesting a signed certificate from the master: Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.181404 52975 manager.go:149] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/k Nov 03 16:37:00 steller.india.com kubelet[52975]: W1103 16:37:00.223876 52975 manager.go:157] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api servic Nov 03 16:37:00 steller.india.com kubelet[52975]: W1103 16:37:00.224005 52975 manager.go:166] unable to connect to CRI-O api service: Get http://%2Fvar%2Frun%2Fcrio.so Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.262573 52975 fs.go:139] Filesystem UUIDs: map[17856e0b-777f-4065-ac97-fb75d7a1e197:/dev/dm-1 2dc6a878- Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.262604 52975 fs.go:140] Filesystem partitions: map[/dev/sdb:{mountpoint:/D major:8 minor:16 fsType:xfs Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.268969 52975 manager.go:216] Machine: {NumCores:56 CpuFrequency:2600000 MemoryCapacity:540743667712 Hu Nov 03 16:37:00 steller.india.com kubelet[52975]: 967295 Mtu:1500} {Name:eno49 MacAddress:14:02:ec:82:57:30 Speed:10000 Mtu:1500} {Name:eno50 MacAddress:14:02:ec:82:57:3 Nov 03 16:37:00 steller.india.com kubelet[52975]: evel:1} {Size:262144 Type:Unified Level:2}]} {Id:13 Threads:[12 40] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Nov 03 16:37:00 steller.india.com kubelet[52975]: s:[26 54] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level: Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.270145 52975 manager.go:222] Version: {KernelVersion:3.10.0-229.14.1.el7.x86_64 ContainerOsVersion:Cen Nov 03 16:37:00 steller.india.com kubelet[52975]: I1103 16:37:00.271263 52975 server.go:422] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaultin Nov 03 16:37:00 steller.india.com kubelet[52975]: error: failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fail-swap-on flag to Nov 03 16:37:00 steller.india.com systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE Nov 03 16:37:00 steller.india.com systemd[1]: Unit kubelet.service entered failed state. Nov 03 16:37:00 steller.india.com systemd[1]: kubelet.service failed.

Answer 1

只需禁用机器中的交换。 须藤交换 -a

Answer 2

一些 Kubernetes 部署需要交换

一般来说， 官方要求是禁用swap。

原因？ Kubernetes 尚不支持。 IMO 他们要求您禁用交换以防止多节点集群工作负载转移出现问题。

但是，我有一个有效的用例 - 我正在开发一个本地产品，linux 发行版，包含在 kubeadm 中。 设计上没有水平缩放。 为了在机会性内存峰值中幸存下来并仍然起作用（但速度很慢），我绝对需要 swap 。

---

要在启用交换的情况下安装kubeadm ：

1. 在/etc/systemd/system/kubelet.service.d/20-allow-swap.conf创建一个文件，内容如下：

    [Service] Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"

2. 跑步

   sudo systemctl daemon-reload

3. 使用标志--ignore-preflight-errors=Swap初始化kubeadm

    kubeadm init --ignore-preflight-errors=Swap

Answer 3

sudo swapoff -a 在重启后不持久。

您可以在重新启动后禁用交换，只需注释掉（在行前添加 #）/etc/fstab 文件中的交换条目。 它会阻止交换分区在重新启动后自动挂载。

脚步：

1. 打开文件 /etc/fstab

2. 在下面找一行

   # 安装期间交换位于 /dev/sda5

   UUID=e7bf7d6e-e90d-4a96-91db-7d5f282f9363 无交换 sw 0 0

3. 用 # 注释掉上面的行并保存。 它应该如下所示

   # 安装期间交换位于 /dev/sda5

   #UUID=e7bf7d6e-e90d-4a96-91db-7d5f282f9363 无交换 sw 0 0

4. 重新启动系统或为当前会话执行“sudo swapoff -a”以避免重新启动。

Answer 4

看起来您在服务器上启用了交换，您可以禁用它并重新运行 init 命令。

error: failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fail-swap-on flag to

以下是在 centos 中禁用交换的步骤https://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-swap-removing.html

Answer 5

如果您在openstack或任何云上运行，请确保允许安全组TCP | TCP / IP中的端口。 入站 6443 * | Kubernetes API服务器

就我而言，这是错误

Answer 6

我猜不同的用户会因为不同的原因遇到这个问题，所以我会在这里添加额外的解释。

最快和最安全的解决方案是禁用交换——就像@Jossef_Harush 共享的链接中提到的那样，这也是 K8S 推荐的。
但正如他也提到的 - 一些工作负载需要更深入的内存管理。

如果您遇到上述错误，因为交换是有意启用的- 我建议在考虑启用交换之前阅读有关驱逐最终用户 Pod 的信息：

如果 kubelet 无法在节点上回收足够的资源，kubelet 将开始驱逐 Pod。

kubelet 首先根据 Pod 对饥饿资源的使用是否超过请求，然后根据优先级，然后根据与 Pod 的调度请求相关的饥饿计算资源的消耗对 Pod 进行逐出排名。

因此，kubelet 按以下顺序排列和驱逐 Pod：

• BestEffort 或 Burstable Pod，其饥饿资源的使用超过其请求。 此类 pod 按优先级排序，然后使用率高于请求。

• 使用率低于请求的有保证的 Pod 和 Burstable Pod 最后被驱逐。 只有在为所有容器指定了请求和限制并且它们相等时，保证 Pod 才能得到保证。 这样的 Pod 保证永远不会因为另一个 Pod 的资源消耗而被驱逐。 如果系统守护进程（例如 kubelet、docker 和 journald）消耗的资源多于通过系统保留或 kube 保留分配保留的资源，并且该节点只有使用少于剩余请求的保证或突发 Pod，则该节点必须选择驱逐这样的 Pod 以保持节点稳定性并限制意外消耗对其他 Pod 的影响。 在这种情况下，它会选择首先驱逐最低优先级的 pod。

确保您还熟悉：

1 ) 3 个 qos 类- 确保您的高优先级工作负载运行有保证（或至少 Burstable）类。

2) Pod 优先级和抢占。