How to get Internal IP of node by hostname

Question

The situation is:

I am trying to deploy HDFS on Kubernetes. I am using DaemonSet to start one datanode on one node and the network mode for the pod is hostNetWork . In HDFS, datanodes and namenodes communicate by hostname. But now, if I add a new node to k8s cluster, existed nodes cannot get the internal IP of the new node by the hostname of the new node.

That is, I want to get node's Internal IP by hostname. Could kube-dns do this now ? Or using some other method to scale nodes for HDFS cluster on k8s.

Thanks a lot!

Answer 1

Reference

Expose Pod Information to Containers Through Environment Variables

You can get the node name, node IP where a pod is deployed as below.

spec:
  containers:
    - name: hdfs-pod
      env:
        - name: MY_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: MY_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_SERVICE_ACCOUNT
          valueFrom:
            fieldRef:
              fieldPath: spec.serviceAccountName

As in DNS for Services and Pods , you can then use the pod IP or name ,and namespace to query the K8S internal DNS.

When enabled, pods are assigned a DNS A record in the form of “pod-ip-address.my-namespace.pod.cluster.local”

---

Statefulset

How about considering using StatefulSet ? Each pod gets a specific name and based on the name, able to assign a specific role eg name node.

Just quickly search and found several articles on HDFS with K8S/StaefulSet.

• https://blog.hasura.io/getting-started-with-hdfs-on-kubernetes-a75325d4178c
• Easiest HDFS cluster in the world with kubernetes
• Repository holding configuration files for running an HDFS cluster in Kubernetes