带有本地主机的Dockerized Kafka TimeoutException

Question

Hi I got an issue with my dockerized kafka server. It does not create messages when send by kafka-console-producer.bat to localhost The eror message is:

ERROR Error when sending message to topic test with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Topic test not present in metadata after 60000 ms.

The zookeeper and kafka servers are running fine and my yml looks like this:

version: '3' 

services:
  zookeeper:
    image: zookeeper 
    ports:
      - 2181:2181
    networks:
      - test
  kafka:
    image: kafka:0.1
    hostname: kafka
    ports:
      - 9092:9092
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://localhost:9092
      - KAFKA_ADVERTISED_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://localhost:9092
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=DOCKER_INTERNAL:PLAINTEXT,LOCALHOST:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=DOCKER_INTERNAL
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - test
networks:
  test:

The kafka dockerfile basically just writes the properties to the config/server.properties of kafka and executes kafka-server-start.sh from the latest kafka distribution 2.12-2.3.0

I created a topic during start up and that is retrievable from zookeeper properly: .\\kafka-topics.bat --describe --topic outbound --zookeeper localhost:2181 Result:

Topic:inbound   PartitionCount:1        ReplicationFactor:1     Configs:
    Topic: inbound  Partition: 0    Leader: 1       Replicas: 1     Isr: 1

But even using this topic to write a message I get the same error as with a new topic.

Maybe someone had a similar issue, Christian

Answer 1

I got it to work :) So the kafka localhost listener has to bind to all interfaces (maybe something to do with the docker network bridge but I am not sure). So the yml property is: KAFKA_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://:9092 KAFKA_ADVERTISED_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://localhost:9092