[英]Why do kubelet failed to get stats from docker (error 500)?
我只是在我的kubelet日志中發現了很多錯誤，如下所示：
Dec 16 15:21:00 kubenode4 kubelet[6851]: I1216 15:21:00.010282 6851 server.go:281] HTTP InternalServerError: Internal Error: unable to find Docker container "d3c1a6936a84028bd420f75c44a13a6ad7c816b651d33877352f2708dccbd943"
Dec 16 15:21:00 kubenode4 kubelet[6851]: I1216 15:21:00.015338 6851 server.go:944] GET /stats/kube-system/kubedash-lm97r/cfa615f2-a405-11e5-8e30-525400ad92ba/kubedash: (10.337027ms) 500
Dec 16 15:21:00 kubenode4 kubelet[6851]: goroutine 2288 [running]:
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/httplog.(*respLogger).WriteHeader(0xc20953cfc0, 0x1f4)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/httplog/log.go:188 +0x9a
Dec 16 15:21:00 kubenode4 kubelet[6851]: net/http.Error(0x7f74e8150938, 0xc20953cfc0, 0xc2088c9e00, 0x72, 0x1f4)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /usr/src/go/src/net/http/server.go:1274 +0x9a
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).error(0xc2080df360, 0x7f74e8150938, 0xc20953cfc0, 0x7f74ebf63dc0, 0xc2089665f0)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:282 +0x213
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).serveStats(0xc2080df360, 0x7f74e8150938, 0xc20953cfc0, 0xc2086c6d00)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:1034 +0x78f
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).handleStats(0xc2080df360, 0x7f74e8150938, 0xc20953cfc0, 0xc2086c6d00)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:440 +0x46
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.*Server.(k8s.io/kubernetes/pkg/kubelet.handleStats)��fm(0x7f74e8150938, 0xc20953cfc0, 0xc2086c6d00)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:154 +0x45
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*httpHandler).ServeHTTP(0xc2081de558, 0x7f74e8150938, 0xc20953cfc0, 0xc2086c6d00)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:275 +0x45
Dec 16 15:21:00 kubenode4 kubelet[6851]: net/http.(*ServeMux).ServeHTTP(0xc2081f5350, 0x7f74e8150938, 0xc20953cfc0, 0xc2086c6d00)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /usr/src/go/src/net/http/server.go:1541 +0x17d
Dec 16 15:21:00 kubenode4 kubelet[6851]: github.com/emicklei/go-restful.Container.ServeHTTP(0x0, 0x0, 0x0, 0xc20824b3e0, 0x2, 0x2, 0xc2081f5350, 0x0, 0x1fc09a0, 0x0, ...)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/emicklei/go-restful/container.go:264 +0x4f
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).ServeHTTP(0xc2080df340, 0x7f74e8150900, 0xc208c38000, 0xc2086c6d00)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:943 +0x162
Dec 16 15:21:00 kubenode4 kubelet[6851]: net/http.serverHandler.ServeHTTP(0xc208045f2 [[Go 1.1 package http] 10.233.64.1:57859]
Dec 16 15:21:00 kubenode4 kubelet[6851]: I1216 15:21:00.010309 6851 server.go:281] HTTP InternalServerError: Internal Error: unable to find Docker container "534ae8d2b4f4e2a111b7c38f9c67b9189f001f72a26fb6e76c3d71e3ec790a01"
Dec 16 15:21:00 kubenode4 kubelet[6851]: I1216 15:21:00.015509 6851 server.go:944] GET /stats/kube-system/influxdb-kj5j2/c92a2514-a405-11e5-8e30-525400ad92ba/influxdb: (10.195837ms) 500
Dec 16 15:21:00 kubenode4 kubelet[6851]: goroutine 2306 [running]:
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/httplog.(*respLogger).WriteHeader(0xc20953d140, 0x1f4)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/httplog/log.go:188 +0x9a
Dec 16 15:21:00 kubenode4 kubelet[6851]: net/http.Error(0x7f74e8150938, 0xc20953d140, 0xc20938f400, 0x72, 0x1f4)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /usr/src/go/src/net/http/server.go:1274 +0x9a
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).error(0xc2080df360, 0x7f74e8150938, 0xc20953d140, 0x7f74ebf63dc0, 0xc208e9c240)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:282 +0x213
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).serveStats(0xc2080df360, 0x7f74e8150938, 0xc20953d140, 0xc2086c7040)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:1034 +0x78f
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).handleStats(0xc2080df360, 0x7f74e8150938, 0xc20953d140, 0xc2086c7040)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:440 +0x46
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.*Server.(k8s.io/kubernetes/pkg/kubelet.handleStats)��fm(0x7f74e8150938, 0xc20953d140, 0xc2086c7040)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:154 +0x45
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*httpHandler).ServeHTTP(0xc2081de558, 0x7f74e8150938, 0xc20953d140, 0xc2086c7040)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:275 +0x45
Dec 16 15:21:00 kubenode4 kubelet[6851]: net/http.(*ServeMux).ServeHTTP(0xc2081f5350, 0x7f74e8150938, 0xc20953d140, 0xc2086c7040)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /usr/src/go/src/net/http/server.go:1541 +0x17d
Dec 16 15:21:00 kubenode4 kubelet[6851]: github.com/emicklei/go-restful.Container.ServeHTTP(0x0, 0x0, 0x0, 0xc20824b3e0, 0x2, 0x2, 0xc2081f5350, 0x0, 0x1fc09a0, 0x0, ...)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/emicklei/go-restful/container.go:264 +0x4f
Dec 16 15:21:00 kubenode4 kubelet[6851]: k8s.io/kubernetes/pkg/kubelet.(*Server).ServeHTTP(0xc2080df340, 0x7f74e8150900, 0xc208c38140, 0xc2086c7040)
Dec 16 15:21:00 kubenode4 kubelet[6851]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server.go:943 +0x162
Dec 16 15:21:00 kubenode4 kubelet[6851]: net/http.serverHandler.ServeHTTP(0xc208045f2 [[Go 1.1 package http] 10.233.64.1:57861]
Dec 16 15:21:00 kubenode4 kubelet[6851]: I1216 15:21:00.016198 6851 server.go:281] HTTP InternalServerError: Internal Error: unable to find Docker container "9db4584cdd04a376db5c43197073e7e1e014d8c3213dbe6ace1e8abf54fe4de2"
Dec 16 15:21:00 kubenode4 kubelet[6851]: I1216 15:21:00.016272 6851 server.go:944] GET /stats/kube-system/influxdb-kj5j2/c92a2514-a405-11e5-8e30-525400ad92ba/grafana: (131.759��s) 500
我使用heapster監視集群，似乎對“ stats”的調用得到了答案，但由於未知原因，它被視為錯誤500。
我在裸機服務器上運行集群。
kubernetes版本：v1.1.3
碼頭工人：1.9.1
操作系統：Debian 8.1
對我來說似乎是個蟲子。 我已經提交了https://github.com/kubernetes/kubernetes/issues/18868
Kubelet 無法獲取“/system.slice/docker.service”的 cgroup 統計信息
[英]Kubelet failed to get cgroup stats for “/system.slice/docker.service”
kubelet 因 kubelet cgroup 驅動程序失敗：“cgroupfs”不同於 docker cgroup 驅動程序：“systemd”
[英]kubelet failed with kubelet cgroup driver: "cgroupfs" is different from docker cgroup driver: "systemd"
當我試圖提取公共圖像時，為什么我從Docker獲得授權錯誤？
[英]Why do I get an authorization error from Docker when I'm trying to pull a public image?
systemctl start kubelet從主節點刪除docker映像
[英]systemctl start kubelet deleting docker images from the master node
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.