How can i access a k8s worker node from a master node without ssh?

Question

I want to build a python script that accesses a kubernetes worker node, lists the network interfaces and captures the traffic of the choosen interface (with tcpdump for exemple) and then store the pcap file somewhere on the master node.

I want to know if it is possible to access a worker node from my master node without ssh? (maybe with a direct k8s-apiserver call ?) If ssh is the only way to access the worker node, how can i make a connection without entring the worker's password (for authentification).

Or maybe there is another way to do this ?

Answer 1

One way to connect nodes, when SSH isn't an option, would be to start some privileged container, that would access your nodes filesystem, disabling pid namespace isolation.

Say I have some node "infra1"

$> kubectl get nodes
infra1     Ready         infra         728d   v1.21.6

I can get in using:

$ kubectl debug node/infra1 -it --image=busybox
Creating debugging pod node-debugger-infra1-s46g6 with container debugger on node infra1.
If you don't see a command prompt, try pressing enter.
/ #
/ # chroot /host
root@infra1:/# 
root@infra1:/# crictl ps | head -2
CONTAINER           IMAGE               CREATED             STATE               
NAME                       ATTEMPT             POD ID
a36863efe2a2e       3fb5cabb64693       4 minutes ago       Running             

The /host being a "volume", sharing my host filesystem with that debug container. Using chroot, you're now working from your node runtime.

$ k get pods 
NAME                                      READY   STATUS    RESTARTS   AGE
node-debugger-infra1-g5nwg                0/1     Error     0          71s
node-debugger-infra1-s46g6                1/1     Running   0          55s

In practice, this is done creating a Pod, such as the following:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubernetes.io/psp: hostaccess
  name: node-debugger-infra1-s46g6
  namespace: default
spec:
  containers:
  - image: busybox
    ...
    volumeMounts:
    - mountPath: /host
      name: host-root
      ...
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  hostIPC: true
  hostNetwork: true
  hostPID: true
  nodeName: infra1
  nodeSelector:
    kubernetes.io/os: linux
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Never
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - operator: Exists
  volumes:
  - hostPath:
      path: /
      type: ""
    name: host-root
  ...

---

Answering your follow-up question, regarding caliXXX interfaces. Those are specific to calico SDN, although the same remarks may apply to other implementations: there's no easy way to resolve pods IPs from those.

We can however inspect pods configuration, figuring out which interface they use:

# crictl pods
...
b693b9ff6487c       3 hours ago         Ready               controller-6b78bff7d9-5b7vr                 metallb-system          2                   (default)

# crictl inspectp b693b9ff6487c | jq '.info.cniResult'
{
  "Interfaces": {
    "cali488d4aeb0e6": {
      "IPConfigs": null,
      "Mac": "",
      "Sandbox": ""
    },
    "eth0": {
      "IPConfigs": [
        {
          "IP": "10.233.105.55",
          "Gateway": ""
        }
      ],
      "Mac": "",
      "Sandbox": ""
    },
    "lo": {
      "IPConfigs": [
        {
          "IP": "127.0.0.1",
          "Gateway": ""
        },
        {
          "IP": "::1",
          "Gateway": ""
        }
      ],
      "Mac": "00:00:00:00:00:00",
      "Sandbox": "/var/run/netns/cni-55322514-e37a-2913-022a-9f7488df8ca5"
    }
  },
  "DNS": [
    {},
    {}
  ],
  "Routes": null
}

Then, resolving the interface name for a given IP, we could so something like:

# MATCH_IP="10\.233\.105\.55"
# crictl pods | awk '/Ready/{print $1}' | while read pod
    do
        crictl inspectp $pod | grep $MATCH_IP >/dev/null 2>&1 || continue
        echo found pod $pod
        crictl inspectp $pod \
           | jq '.info.cniResult.Interfaces | with_entries(select(.key|match("cali"))) | to_entries[] | .key'
        break
    done
found pod b693b9ff6487c
"cali488d4aeb0e6"