连接到在 docker 容器中运行的 Kafka 时出错

Question

我已经为我的 spring 基于引导的库配置了以下 Kafka 属性，这些库捆绑在部署到Wildfly的EAR的lib目录中。 通过从类路径（ WEB-INF/classes ）加载 porperty 文件，我能够成功启动 spring 组件

spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.gson.GsonAutoConfiguration,org.springframework.boot.autoconfigure.jms.artemis.ArtemisAutoConfiguration,\
org.springframework.boot.autoconfigure.data.web.SpringDataWebAutoConfiguration
spring.kafka.admin.client-id=iris-admin-local
spring.kafka.producer.client-id=iris-producer-local
spring.kafka.producer.retries=3
spring.kafka.producer.properties.max.block.ms=2000
spring.kafka.producer.bootstrap-servers=127.0.0.1:19092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer
foo.app.kafka.executor.core-pool-size=10
foo.app.kafka.executor.max-pool-size=500
foo.app.kafka.executor.queue-capacity=1000

我通过 docker compose 运行 Kafka 和 zookeeper，容器分别映射到主机端口12181和19092 。 发布失败并出现错误

19:37:42,914 ERROR [org.springframework.kafka.support.LoggingProducerListener] (swiftalker-3) Exception thrown when sending a message with key='543507' and payload='com.foo.app.kanban.defect.entity.KanbanDefect@84b13' to topic alm_swift-alm:: org.apache.kafka.common.errors.TimeoutException: Topic alm_swift-alm not present in metadata after 2000 ms.

19:37:43,124 WARN  [org.apache.kafka.clients.NetworkClient] (kafka-producer-network-thread | iris-producer-local-1) [Producer clientId=iris-producer-local-1] Error connecting to node 6be446692a1f:9092 (id: 1001 rack: null): java.net.UnknownHostException: 6be446692a1f
    at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
    at java.net.InetAddress.getAllByName(InetAddress.java:1193)
    at java.net.InetAddress.getAllByName(InetAddress.java:1127)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:955)
    at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:73)
    at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1128)
    at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1016)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:324)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
    at java.lang.Thread.run(Thread.java:748)

现在这是在提供spring.kafka.producer.bootstrap-servers=127.0.0.1:19092属性之后。 不过有趣的是

CONTAINER ID   NAMES                PORTS                                                                          CREATED          STATUS
2133c81ed51d   mongo                0.0.0.0:23556->27017/tcp, 0.0.0.0:23557->27018/tcp, 0.0.0.0:23558->27019/tcp   29 minutes ago   Up 29 minutes
f18b86d8739e   kafka-ui             0.0.0.0:18080->8080/tcp                                                        29 minutes ago   Up 29 minutes
6be446692a1f   kafka                0.0.0.0:19092->9092/tcp                                                        29 minutes ago   Up 29 minutes
873304e1e6a0   zookeeper            2181/tcp, 2888/tcp, 3888/tcp, 8080/tcp                                         29 minutes ago   Up 29 minutes

Wildfly 服务器错误日志显示该应用程序实际上是通过其容器 ID 连接到 docker 容器，即

6be446692a1f   kafka                0.0.0.0:19092->9092/tcp

从docker ps -a output 和

Error connecting to node 6be446692a1f:9092 (id: 1001 rack: null): java.net.UnknownHostException: 6be446692a1f

我很困惑 spring 引导代码如何，尽管配置属性在 localhost 和映射端口19092上引用服务器，但设法在其 ID 和默认端口上找到 docker 容器，然后尝试连接到它？ 我该如何解决？

更新：docker 组成

version: '3'

networks:
  app-tier:
    driver: bridge

services:
  zookeeper:
    image: 'docker.io/bitnami/zookeeper:3-debian-10'
    container_name: 'zookeeper'
    networks:
      - app-tier
    volumes:
      - 'zookeeper_data:/bitnami'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'docker.io/bitnami/kafka:2-debian-10'
    container_name: 'kafka'
    ports:
      - 19092:9092
    networks:
      - app-tier
    volumes:
      - 'kafka_data:/bitnami'
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper
  database:
    image: 'mongo'
    container_name: 'mongo'
    environment:
      - MONGO_INITDB_DATABASE='swiftalk_db'
    networks:
      - app-tier
    ports:
      - 23556-23558:27017-27019
    depends_on:
      - kafka
  kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - 18080:8080
    networks:
      - app-tier
    volumes: 
      - 'mongo_data:/data/db'
    depends_on:
      - kafka
    environment:
      - KAFKA_CLUSTERS_0_NAME=local
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
      - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181

volumes:
  zookeeper_data:
    driver: local
  kafka_data:
    driver: local
  mongo_data:
    driver: local

Answer 1

您尚未共享您的 Docker Compose，因此我无法为您提供具体的修复方法，但本质上您需要正确配置您的广告侦听器。 这是代理提供给客户端的值，告诉它在进行后续连接时在哪里可以找到它。

详细信息： https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/