Apache Kafka和Zookeeper連接問題

Question

我正在嘗試使用 Docker 群創建多節點 Apache Kafka 集群。 為此，我使用以下 docker-compose.yml

version: '3.2'
services:
   zookeeper:
      image: wurstmeister/zookeeper
      ports:
         - "2181:2181"

   kafka:
      image: wurstmeister/kafka:latest
      deploy:
         mode: global
      ports:
         - target: 9094
           published: 9094
           protocol: tcp
           mode: host
      environment:
         HOSTNAME_COMMAND: "docker info | grep ^Name: | cut -d' ' -f 2" # Normal instances
         KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
         KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
         KAFKA_ADVERTISED_PROTOCOL_NAME: OUTSIDE
         KAFKA_ADVERTISED_PORT: 9094
         KAFKA_PROTOCOL_NAME: INSIDE
         KAFKA_PORT: 9092
         KAFKA_CREATE_TOPICS: myTopic:1:1
      volumes:
         - /var/run/docker.sock:/var/run/docker.sock

但是，當創建 kafka_kafka 服務時，我遇到以下錯誤：

java.lang.IllegalArgumentException: Error creating broker listeners from 'PLAINTEXT://ugur:9094': No security protocol defined for listener PLAINTEXT
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:312)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.server.KafkaConfig.advertisedListeners(KafkaConfig.scala:1300)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1360)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1338)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1028)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:1008)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.Kafka$.main(Kafka.scala:59)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.Kafka.main(Kafka.scala)
kafka_kafka.0.wx4chyussuuc@ugur    | Caused by: java.lang.IllegalArgumentException: No security protocol defined for listener PLAINTEXT
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.cluster.EndPoint$$anonfun$securityProtocol$1$1.apply(EndPoint.scala:48)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.cluster.EndPoint$$anonfun$securityProtocol$1$1.apply(EndPoint.scala:48)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.AbstractMap.getOrElse(Map.scala:59)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.cluster.EndPoint$.securityProtocol$1(EndPoint.scala:47)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.cluster.EndPoint$.createEndPoint(EndPoint.scala:56)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.utils.CoreUtils$$anonfun$6.apply(CoreUtils.scala:309)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.utils.CoreUtils$$anonfun$6.apply(CoreUtils.scala:309)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
kafka_kafka.0.wx4chyussuuc@ugur    |    at scala.collection.AbstractTraversable.map(Traversable.scala:104)
kafka_kafka.0.wx4chyussuuc@ugur    |    at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:309)

我還打開了必要的端口，這些端口是這樣的：

采取行動從

--------------

9094 允許在任何地方
9092 允許在任何地方
8081 允許在任何地方
8080 允許在任何地方
第2377章
7946 允許在任何地方
4789 允許在任何地方
9094 (v6) 允許在任何地方 (v6)
9092 (v6) 允許在任何地方 (v6)
8081 (v6) 允許在任何地方 (v6)
8080 (v6) 允許在任何地方 (v6)
2377 (v6) 允許在任何地方 (v6)
7946 (v6) 允許在任何地方 (v6)
4789 (v6) 允許在任何地方 (v6)

Answer 1

您是否嘗試過使用GitHub上提供的docker-compose-swarm.yml？

https://github.com/wurstmeister/kafka-docker/blob/master/docker-compose-swarm.yml

也許缺少的屬性是關鍵：

KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://_{HOSTNAME_COMMAND}:9094

Answer 2

我添加了以下選項：

  KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
  KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
  KAFKA_LISTENERS: INTERNAL://:29092,EXTERNAL://:9092
  KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://localhost:9092
  ALLOW_PLAINTEXT_LISTENER: "yes"

顯然純文本偵聽器並不安全，但它應該可以幫助您入門。