Kafka SASL_SSL 在指定的 JAAS 配置文件中找不到名為“Client”的 JAAS 配置部分

Question

我正在嘗試在我的 kafka 代理中使用 SASL/PLAIN 激活身份驗證。

JAAS配置文件如下

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret"
    user_alice="alice-secret";
};

我使用以下命令啟動 kafka 服務

export KAFKA_OPTS="-Djava.security.auth.login.config=<PATH>kafka_server_jaas.conf
/bin/kafka-server-start.sh /config/server.properties

kafka 服務沒有正常啟動，我在日志中得到了這些錯誤

javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/home/kafka/kafka/config/kafka_server_jaas.conf'.
    at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:189)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1161)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1210)
[2022-03-16 12:13:16,587] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,588] ERROR [ZooKeeperClient Kafka server] Auth failed, initialized=false connectionState=CONNECTING (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 12:13:16,592] INFO Socket connection established, initiating session, client: /127.0.0.1:46706, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,611] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x100002dd98c0000, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,612] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 12:13:16,752] INFO [feature-zk-node-event-process-thread]: Starting (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2022-03-16 12:13:16,786] INFO Feature ZK node at path: /feature does not exist (kafka.server.FinalizedFeatureChangeListener)
[2022-03-16 12:13:16,788] INFO Cleared cache (kafka.server.FinalizedFeatureCache)
[2022-03-16 12:13:16,957] INFO Cluster ID = 6WTadNCMRAW4dHoc_JUnIg (kafka.server.KafkaServer)
[2022-03-16 12:13:16,968] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID 6WTadNCMRAW4dHoc_JUnIg doesn't match stored clusterId Some(RJXzPwJeRfawIa_yA0B26A) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
    at kafka.server.KafkaServer.startup(KafkaServer.scala:228)
    at kafka.Kafka$.main(Kafka.scala:109)
    at kafka.Kafka.main(Kafka.scala)

我已經將以下行添加到server.properties

listeners=SASL_SSL://localhost:9092
security.protocol=SASL_SSL
security.inter.broker.protocol=SASL_SSL
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN

Answer 1

[2022-03-16 12:13:16,968] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID 6WTadNCMRAW4dHoc_JUnIg doesn't match stored clusterId Some(RJXzPwJeRfawIa_yA0B26A) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
    at kafka.server.KafkaServer.startup(KafkaServer.scala:228)
    at kafka.Kafka$.main(Kafka.scala:109)
    at kafka.Kafka.main(Kafka.scala)

當代理的 Zookeeper 和 Kafka 數據目錄中存儲的集群 ID 不匹配時，會出現此問題。

在這種情況下，代理的集群 ID 存儲在

• Zookeeper 數據為6WTadNCMRAW4dHoc_JUnIg
• Kafka meta.properties 是RJXzPwJeRfawIa_yA0B26A

原因： Zookeeper 數據目錄被刪除。

刪除 Zookeeper dataDir並重新啟動 Zookeeper 和 Kafka 服務將不起作用。 因為 Zookeeper 創建一個新的 Cluster ID 並在它注冊時將其分配給 broker，如果還沒有條目的話。 這個新的集群 ID 將與meta.properties中的不同。

可以按照以下步驟解決此問題

• 同時刪除 Kafka log.dirs和 Zookeeper dataDir - 導致數據丟失； Kafka和Zookeeper服務都需要重啟
• 刪除 Kafka meta.properties目錄中的log.dirs - 不會丟失數據； 無論如何都需要啟動kafka服務
• 使用存儲在 Zookeeper 數據中的值更新meta.properties中的集群 ID； 在這種情況下，將RJXzPwJeRfawIa_yA0B26A替換為6WTadNCMRAW4dHoc_JUnIg - 無數據丟失； 無論如何都需要啟動kafka服務

javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file

Client部分用於驗證與 ZooKeeper 的 SASL 連接。 上面的錯誤javax.security.auth.login.LoginException是一個警告，如果 Zookeeper 允許，Kafka 將在沒有 SASL 身份驗證的情況下連接到 Zookeeper 服務器。

[2022-03-16 12:13:16,587] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,588] ERROR [ZooKeeperClient Kafka server] Auth failed, initialized=false connectionState=CONNECTING (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 12:13:16,592] INFO Socket connection established, initiating session, client: /127.0.0.1:46706, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,611] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x100002dd98c0000, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)

KafkaServer 部分由代理使用，並為代理間連接提供 SASL 配置選項。 代理使用用戶名和密碼來啟動與其他代理的連接。 屬性集 user_username 定義了所有用戶連接到代理的密碼。