Cannot connect to other container in kubernetes pod through localhost

Question

I'm trying to launch docker-in-docker so that I can connect to it in a different container in the same Kubernetes pod. I am using the following yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: {{job_name}}
  labels:
    taskType: store_v510
    taskName: {{task_name}}
spec:
  template:
    spec:
      activeDeadlineSeconds: 86400
      volumes:
        - name: hdfs-volume
          configMap:
            name: hdfs-config
            items:
              - key: core-site.xml
                path: core-site.xml
              - key: hadoop-env.sh
                path: hadoop-env.sh
              - key: hdfs-site.xml
                path: hdfs-site.xml
              - key: log4j.properties
                path: log4j.properties
        - name: download-dir
          hostPath:
              path: /tmp/store_pipeline
        - name: docker-graph-storage 
          emptyDir: {}
      containers:
        - name: {{job_name}}
          envFrom:
          - configMapRef:
              name: job-configs
          env: 
          - name: DOCKER_HOST 
            value: tcp://localhost:2375
          volumeMounts: 
          - mountPath: /tmp/store_pipeline
            name: download-dir
          image: registry.com.cn/com/store_pipeline:0.1
          imagePullPolicy: Always
          resources:
            requests:
              cpu: 2
              memory: 32Gi
              nvidia.com/gpu: 0
            limits:
              cpu: 2
              memory: 40Gi
              nvidia.com/gpu: 0
          command: ["bash", "run_store_pipeline.sh", "--data_dir", "{{data_dir}}", "--config_dir", "{{config_dir}}", "--output_dir", "{{output_dir}}"]
        - name: dind-daemon 
          image: registry.com.cn/com/1.12.6-dind
          resources: 
              requests: 
                  cpu: 20m 
                  memory: 512Mi 
          securityContext: 
              privileged: true 
          volumeMounts: 
            - name: docker-graph-storage 
              mountPath: /var/lib/docker 
          command: []

Based on the documentation I see, I should be able to connect to the docker-in-docker docker daemon by setting DOCKER_HOST = tcp://localhost:2375. However, I'm not able to do so.

I notice that if I test it locally and run these two containers in the same docker network, I'm able to connect them by specifying the actual container name (ie tcp://dind-daemon:2375). Thanks!

Answer 1

Your setup seems to be correct. I assume that the docker image is comparable to a current docker image from the public registry.

That docker-in-docker usually listens only for TLS secured requests on port 2376.

If you want to be able to use it on 2375 without mutual TLS authentication you need to set the environment variable DOCKER_TLS_CERTDIR to an empty value.