kubernetes节点未注册

Question

I'm trying to kubernetes 1.5.2 on Container Linux by CoreOS alpha (1284.2.0) using rkt.

I have two coreos servers, one (controller+worker) with hostname coreos-2.tux-in.com , the 2nd one will be a work with hostname coreos-3.tux-in.com .

for now I'm installing the controller+worker on coreos-2.tux-in.com .

in general I followed the instructions in https://coreos.com/kubernetes/docs/latest/ and aded some modifications.

instead of using deprecated --api-server parameter I use kubeconfig.

the problem that I'm having is that the kube-proxy pod fails with the following error messages:

Jan 14 23:27:34 coreos-2.tux-in.com rkt[11555]: [  220.477192] kube-proxy[5]: E0114 23:27:34.900184       5 server.go:421] Can't get Node "coreos-2.tux-in.com", assuming iptables proxy, err: nodes "coreos-2.tux-in.com" not found
Jan 14 23:27:34 coreos-2.tux-in.com rkt[11555]: [  220.479181] kube-proxy[5]: I0114 23:27:34.902440       5 server.go:215] Using iptables Proxier.
Jan 14 23:27:34 coreos-2.tux-in.com rkt[11555]: [  220.480503] kube-proxy[5]: W0114 23:27:34.903771       5 server.go:468] Failed to retrieve node info: nodes "coreos-2.tux-in.com" not found
Jan 14 23:27:34 coreos-2.tux-in.com rkt[11555]: [  220.481175] kube-proxy[5]: F0114 23:27:34.903829       5 server.go:222] Unable to create proxier: can't set sysctl net/ipv4/conf/all/route_localnet: open /proc/sys/net/ipv4/conf/all/route_localnet: read-only file system

the kubeconfig is located at /etc/kubernetes/controller-kubeconfig.yaml with the following:

apiVersion: v1
kind: Config
clusters:
- cluster:
    server: http://127.0.0.1:8080
  name: tuxin-coreos-cluster
contexts:
- context:
    cluster: tuxin-coreos-cluster
  name: tuxin-coreos-context
kind: Config
preferences:
  colors: true
users:
- name: kubelet
  user:
    client-certificate: /etc/kubernetes/ssl/apiserver.pem
    client-key: /etc/kubernetes/ssl/apiserver-key.pem
current-context: tuxin-coreos-context

this is the manifest for kube-apisever:

apiVersion: v1
kind: Pod
metadata:
  name: kube-apiserver
  namespace: kube-system
spec:
  hostNetwork: true
  containers:
  - name: kube-apiserver
    image: quay.io/coreos/hyperkube:v1.5.2_coreos.0
    command:
    - /hyperkube
    - apiserver
    - --bind-address=0.0.0.0
    - --etcd-servers=http://127.0.0.1:4001
    - --allow-privileged=true
    - --service-cluster-ip-range=10.3.0.0/24
    - --secure-port=443
    - --advertise-address=10.79.218.2
    - --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota
    - --tls-cert-file=/etc/kubernetes/ssl/apiserver.pem
    - --tls-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem
    - --client-ca-file=/etc/kubernetes/ssl/ca.pem
    - --service-account-key-file=/etc/kubernetes/ssl/apiserver-key.pem
    - --runtime-config=extensions/v1beta1/networkpolicies=true
    - --anonymous-auth=false
    livenessProbe:
      httpGet:
        host: 127.0.0.1
        port: 8080
        path: /healthz
      initialDelaySeconds: 15
      timeoutSeconds: 15
    ports:
    - containerPort: 443
      hostPort: 443
      name: https
    - containerPort: 8080
      hostPort: 8080
      name: local
    volumeMounts:
    - mountPath: /etc/kubernetes/ssl
      name: ssl-certs-kubernetes
      readOnly: true
    - mountPath: /etc/ssl/certs
      name: ssl-certs-host
      readOnly: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/ssl
    name: ssl-certs-kubernetes
  - hostPath:
      path: /usr/share/ca-certificates
    name: ssl-certs-host

and this is the manifest for kube-proxy:

apiVersion: v1
kind: Pod
metadata:
  name: kube-proxy
  namespace: kube-system
spec:
  hostNetwork: true
  containers:
  - name: kube-proxy
    image: quay.io/coreos/hyperkube:v1.5.2_coreos.0
    command:
    - /hyperkube
    - proxy
    - --kubeconfig=/etc/kubernetes/controller-kubeconfig.yaml
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /etc/kubernetes/controller-kubeconfig.yaml
      name: "kubeconfig"
      readOnly: true
    - mountPath: /etc/kubernetes/ssl
      name: "etc-kube-ssl"
      readOnly: true
    - mountPath: /var/run/dbus
      name: dbus
      readOnly: false
  volumes:
  - name: "ssl-certs"
    hostPath:
      path: "/usr/share/ca-certificates"
  - name: "kubeconfig"
    hostPath:
      path: "/etc/kubernetes/controller-kubeconfig.yaml"
  - name: "etc-kube-ssl"
    hostPath:
      path: "/etc/kubernetes/ssl"
  - hostPath:
      path: /var/run/dbus
    name: dbus

/etc/kubernetes/manifests also includes canal, kube-controller-manager, kube-scheduler and kubernetes-dashboard.

I have kubectl on my desktop configured with the following at ~/.kube/config :

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/ufk/Projects/tuxin-coreos/kubernetes/certs/ca.pem
    server: https://coreos-2.tux-in.com
  name: tuxin-coreos-cluster
contexts:
- context:
    cluster: tuxin-coreos-cluster
    user: default-admin
  name: tuxin-coreos-context
current-context: tuxin-coreos-context
kind: Config
preferences: {}
users:
- name: default-admin
  user:
    username: kubelet
    client-certificate: /Users/ufk/Projects/tuxin-coreos/kubernetes/certs/client.pem
    client-key: /Users/ufk/Projects/tuxin-coreos/kubernetes/certs/client-key.pem

and when I execute kubectl get nodes I get No resources found.

so somehow the current node is not registered...

this is my kubelet.service file:

[Service]
Environment=KUBELET_IMAGE_TAG=v1.5.2_coreos.0
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
  --volume var-log,kind=host,source=/var/log \
  --mount volume=var-log,target=/var/log \
  --volume dns,kind=host,source=/etc/resolv.conf \
  --mount volume=dns,target=/etc/resolv.conf \
  --volume cni-bin,kind=host,source=/opt/cni/bin \
  --mount volume=cni-bin,target=/opt/cni/bin \
  --volume rkt,kind=host,source=/opt/bin/host-rkt \
  --mount volume=rkt,target=/usr/bin/rkt \
  --volume var-lib-rkt,kind=host,source=/var/lib/rkt \
  --mount volume=var-lib-rkt,target=/var/lib/rkt \
  --volume stage,kind=host,source=/tmp \
  --mount volume=stage,target=/tmp"
ExecStartPre=/usr/bin/mkdir -p /opt/cni/bin
ExecStartPre=/usr/bin/mkdir -p /etc/kubernetes/manifests
ExecStartPre=/usr/bin/mkdir -p /var/log/containers
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
ExecStart=/usr/lib/coreos/kubelet-wrapper \
  --kubeconfig=/etc/kubernetes/controller-kubeconfig.yaml \
  --register-schedulable=false \
  --network-plugin=cni \
  --container-runtime=rkt \
  --rkt-path=/usr/bin/rkt \
  --allow-privileged=true \
  --pod-manifest-path=/etc/kubernetes/manifests \
  --hostname-override=coreos-2.tux-in.com \
  --cluster_dns=10.3.0.10 \
  --cluster_domain=cluster.local
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

I have --hostname-override=coreos-2.tux-in.com set, so I guess it's supposed to register the node but doesn't.

what do I do from here?

Answer 1

I needed to add --require-kubeconfig parameter to the kubelet-wrappper execution of kubelet.service . this tells kubelet to configure api server from the kubeconfig file.