docker-compose command giving error in kafka-zookeeper setup

Question

I am trying to synchronize kafka with zookeeper and for this I have two files kept on following path:

root@sevenos:/home/sevenos/Downloads/dockerrun# ls -l
total 80
-rw-rw-r-- 1 sevenos sevenos   816 Apr 24 17:12 docker-compose.yml
drwxr-xr-x 3 root    root     4096 Apr 24 15:35 scripts
-rwxrwxr-x 1 sevenos sevenos 55842 Apr 24 17:01 wait-for.sh

wait-for.sh is script taken from Eficode which checks the availability of host and port and it is running fine locally. The docker-compose file consist of following things:

    version: "2.1"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:4.0.0
    restart: always
    hostname: host
    container_name: zookeeper
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
    volumes:
      - $PWD/scripts/security:/etc/kafka/secrets

    ports:
      - "32181:32181"

  kafka:
    image: modkafka
    hostname: host
    container_name: kafka
    depends_on:
      - zookeeper
    command: sh -c '/subdir/wait-for.sh localhost:32181'
    volumes:
      - $PWD/scripts/security:/etc/kafka/secrets
      - ./subdir:/subdir
    ports:
      - "29092:29092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "localhost:32181"
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

when i am triggering this compose file,kafka is getting triggered before and i am getting following error:

kafka        |   export KAFKA_LISTENERS
kafka        |   KAFKA_LISTENERS=$(cub listeners "$KAFKA_ADVERTISED_LISTENERS")
kafka        | fi
kafka        | + [[ -z '' ]]
kafka        | + export KAFKA_LISTENERS
kafka        | cub listeners "$KAFKA_ADVERTISED_LISTENERS"
kafka        | ++ cub listeners 'PLAINTEXT://localhost:29092"'
zookeeper    | sh: 1: ./wait-for: not found
zookeeper    | sh: 1: ./wait-for: not found
zookeeper    | sh: 1: ./wait-for: not found
kafka        | + KAFKA_LISTENERS='PLAINTEXT://0.0.0.0:29092"'
kafka        | 
kafka        | dub path /etc/kafka/ writable
kafka        | + dub path /etc/kafka/ writable
zookeeper exited with code 127

UPDATE

I have created new dockerfile for kafka mentioning following things:

FROM confluentinc/cp-kafka:4.0.0
RUN mkdir subdir

and build it in following way:

docker build -t modkafka .

Issue still persist.

Doing bash into modkafka results in following output:

root@sevenos:/home/sevenos/Downloads/dockerrun# docker exec -it kafka bash
root@sevenos:/# ls
bin   dev  home  lib64  mnt  proc  run   srv     sys  usr
boot  etc  lib   media  opt  root  sbin  subdir  tmp  var
root@sevenos:/# cd subdir

Updated docker-compose.yml file :

version: "2.1"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:4.0.0
    restart: always
    hostname: host
    container_name: zookeeper
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
    volumes:
      - $PWD/scripts/security:/etc/kafka/secrets

    ports:
      - "32181:32181"

  kafka:
    image: modkafka
    hostname: host
    container_name: kafka
    depends_on:
      - zookeeper
    command: sh -c '/subdir/wait-for.sh localhost:32181'
    volumes:
      - $PWD/scripts/security:/etc/kafka/secrets
      - ./subdir:/subdir
    ports:
      - "29092:29092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "localhost:32181"
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

Answer 1

You need to include the script wait-for.sh into the containers.

For this you can mount a directory that contains the script inside the container. I recommend to move the wait-for.sh into a directory, just to be sure that the container doesn't access anything else from the hosts. Let's suppose that you move the script to subdir , then your compose file should look like this:

version: "2.1"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:4.0.0
    restart: always
    hostname: host
    container_name: zookeeper
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
    volumes:
      - $PWD/scripts/security:/etc/kafka/secrets
    ports:
      - "32181:32181"
  kafka:
    image: confluentinc/cp-kafka:4.0.0
    hostname: host
    container_name: kafka
    depends_on:
      - zookeeper
    volumes:
      - $PWD/scripts/security:/etc/kafka/secrets
      - ./subdir:/subdir
    ports:
      - "29092:29092"
    command: sh -c '/subdir/wait-for.sh localhost:32181'
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "localhost:32181"
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1