使用 SASL_PLAINTEXT 的 Kafka 身份验证失败

Question

这是日志：

kafka 16:54:47.56 
kafka 16:54:47.57 Welcome to the Bitnami kafka container
kafka 16:54:47.57 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-kafka
kafka 16:54:47.57 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-kafka/issues
kafka 16:54:47.57 
kafka 16:54:47.57 INFO  ==> ** Starting Kafka setup **
kafka 16:54:47.62 DEBUG ==> Validating settings in KAFKA_* env vars...
kafka 16:54:47.64 WARN  ==> You set the environment variable ALLOW_PLAINTEXT_LISTENER=yes. For safety reasons, do not use this flag in a production environment.
kafka 16:54:47.64 INFO  ==> Initializing Kafka...
kafka 16:54:47.65 INFO  ==> No injected configuration files found, creating default config files
kafka 16:54:47.89 INFO  ==> Configuring Kafka for inter-broker communications with SASL_PLAINTEXT authentication.
kafka 16:54:47.89 INFO  ==> Configuring Kafka for client communications with SASL_PLAINTEXT authentication.
kafka 16:54:47.91 INFO  ==> Generating JAAS authentication file
kafka 16:54:47.93 INFO  ==> ** Kafka setup finished! **

kafka 16:54:47.95 INFO  ==> ** Starting Kafka **
[2022-05-29 16:54:49,343] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2022-05-29 16:54:49,988] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2022-05-29 16:54:50,157] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2022-05-29 16:54:50,163] INFO starting (kafka.server.KafkaServer)
[2022-05-29 16:54:50,164] INFO Connecting to zookeeper on sharif-zookeeper (kafka.server.KafkaServer)
[2022-05-29 16:54:50,188] INFO [ZooKeeperClient Kafka server] Initializing a new session to sharif-zookeeper. (kafka.zookeeper.ZooKeeperClient)
[2022-05-29 16:54:50,194] INFO Client environment:zookeeper.version=3.5.9-83df9301aa5c2a5d284a9940177808c01bc35cef, built on 01/06/2021 20:03 GMT (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,196] INFO Client environment:host.name=sharif-kafka-0.sharif-kafka-headless.default.svc.cluster.local (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,196] INFO Client environment:java.version=11.0.12 (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,196] INFO Client environment:java.vendor=BellSoft (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,196] INFO Client environment:java.home=/opt/bitnami/java (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,197] INFO Client environment:java.class.path=/opt/bitnami/kafka/bin/../libs/activation-1.1.1.jar:/opt/bitnami/kafka/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/bitnami/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/bitnami/kafka/bin/../libs/commons-cli-1.4.jar:/opt/bitnami/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/bitnami/kafka/bin/../libs/connect-api-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-basic-auth-extension-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-file-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-json-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-mirror-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-mirror-client-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-runtime-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/connect-transforms-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/hk2-api-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/hk2-locator-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/hk2-utils-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/jackson-annotations-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-core-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-databind-2.10.5.1.jar:/opt/bitnami/kafka/bin/../libs/jackson-dataformat-csv-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-datatype-jdk8-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-jaxrs-base-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-jaxrs-json-provider-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-module-jaxb-annotations-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-module-paranamer-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jackson-module-scala_2.12-2.10.5.jar:/opt/bitnami/kafka/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/bitnami/kafka/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/bitnami/kafka/bin/../libs/jakarta.inject-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/bitnami/kafka/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/opt/bitnami/kafka/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/bitnami/kafka/bin/../libs/javassist-3.27.0-GA.jar:/opt/bitnami/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/bitnami/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/bitnami/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/bitnami/kafka/bin/../libs/jersey-client-2.31.jar:/opt/bitnami/kafka/bin/../libs/jersey-common-2.31.jar:/opt/bitnami/kafka/bin/../libs/jersey-container-servlet-2.31.jar:/opt/bitnami/kafka/bin/../libs/jersey-container-servlet-core-2.31.jar:/opt/bitnami/kafka/bin/../libs/jersey-hk2-2.31.jar:/opt/bitnami/kafka/bin/../libs/jersey-media-jaxb-2.31.jar:/opt/bitnami/kafka/bin/../libs/jersey-server-2.31.jar:/opt/bitnami/kafka/bin/../libs/jetty-client-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-continuation-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-http-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-io-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-security-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-server-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-servlet-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-servlets-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-util-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jetty-util-ajax-9.4.39.v20210325.jar:/opt/bitnami/kafka/bin/../libs/jline-3.12.1.jar:/opt/bitnami/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/bitnami/kafka/bin/../libs/kafka-clients-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-log4j-appender-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-metadata-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-raft-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-shell-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-examples-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-scala_2.12-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-test-utils-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka-tools-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/kafka_2.12-2.8.0-sources.jar:/opt/bitnami/kafka/bin/../libs/kafka_2.12-2.8.0.jar:/opt/bitnami/kafka/bin/../libs/log4j-1.2.17.jar:/opt/bitnami/kafka/bin/../libs/lz4-java-1.7.1.jar:/opt/bitnami/kafka/bin/../libs/maven-artifact-3.6.3.jar:/opt/bitnami/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/bitnami/kafka/bin/../libs/netty-buffer-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-codec-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-common-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-handler-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-resolver-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-native-epoll-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-native-unix-common-4.1.62.Final.jar:/opt/bitnami/kafka/bin/../libs/osgi-resource-locator-1.0.3.jar:/opt/bitnami/kafka/bin/../libs/paranamer-2.8.jar:/opt/bitnami/kafka/bin/../libs/plexus-utils-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/reflections-0.9.12.jar:/opt/bitnami/kafka/bin/../libs/rocksdbjni-5.18.4.jar:/opt/bitnami/kafka/bin/../libs/scala-collection-compat_2.12-2.3.0.jar:/opt/bitnami/kafka/bin/../libs/scala-java8-compat_2.12-0.9.1.jar:/opt/bitnami/kafka/bin/../libs/scala-library-2.12.13.jar:/opt/bitnami/kafka/bin/../libs/scala-logging_2.12-3.9.2.jar:/opt/bitnami/kafka/bin/../libs/scala-reflect-2.12.13.jar:/opt/bitnami/kafka/bin/../libs/slf4j-api-1.7.30.jar:/opt/bitnami/kafka/bin/../libs/slf4j-log4j12-1.7.30.jar:/opt/bitnami/kafka/bin/../libs/snappy-java-1.1.8.1.jar:/opt/bitnami/kafka/bin/../libs/zookeeper-3.5.9.jar:/opt/bitnami/kafka/bin/../libs/zookeeper-jute-3.5.9.jar:/opt/bitnami/kafka/bin/../libs/zstd-jni-1.4.9-1.jar (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,197] INFO Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,198] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,198] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,198] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,198] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,198] INFO Client environment:os.version=5.4.190-107.353.amzn2.x86_64 (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,199] INFO Client environment:user.name=? (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,199] INFO Client environment:user.home=? (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,199] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,199] INFO Client environment:os.memory.free=1011MB (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,200] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,200] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,203] INFO Initiating client connection, connectString=sharif-zookeeper sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@447a020 (org.apache.zookeeper.ZooKeeper)
[2022-05-29 16:54:50,210] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2022-05-29 16:54:50,216] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2022-05-29 16:54:50,218] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2022-05-29 16:54:50,367] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2022-05-29 16:54:50,371] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2022-05-29 16:54:50,381] INFO Opening socket connection to server sharif-zookeeper/10.100.190.137:2181. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2022-05-29 16:54:50,389] INFO Socket connection established, initiating session, client: /192.168.34.166:57652, server: sharif-zookeeper/10.100.190.137:2181 (org.apache.zookeeper.ClientCnxn)
[2022-05-29 16:54:50,398] INFO Session establishment complete on server sharif-zookeeper/10.100.190.137:2181, sessionid = 0x100003ea9fd0008, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)
[2022-05-29 16:54:50,406] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2022-05-29 16:54:50,420] ERROR SASL authentication failed using login context 'Client' with exception: {} (org.apache.zookeeper.client.ZooKeeperSaslClient)
javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null.
    at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:312)
    at org.apache.zookeeper.client.ZooKeeperSaslClient.respondToServer(ZooKeeperSaslClient.java:275)
    at org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:882)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:103)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:365)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
[2022-05-29 16:54:50,431] INFO Unable to read additional data from server sessionid 0x100003ea9fd0008, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2022-05-29 16:54:50,430] ERROR [ZooKeeperClient Kafka server] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2022-05-29 16:54:50,452] INFO EventThread shut down for session: 0x100003ea9fd0008 (org.apache.zookeeper.ClientCnxn)

我在 kafka 服务器中的 kafka_jaas 文件：

KafkaClient {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="user_kafka"
   password="secret";
   };
KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="kafka"
   password="CMwvKfeVJociGkSToMZQ"
   user_kafka="CMwvKfeVJociGkSToMZQ"
   user_user_kafka="secret";
   org.apache.kafka.common.security.scram.ScramLoginModule required;
   };
Client {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="kafka"
   password="2q0T4HFZwa21DCRlfqxX";
   };

我在 ZooKeeper 服务器中的 zoo_jaas 文件：

Client {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    username="kafka"
    password="secret";
};
Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_kafka="secret";
};

任何帮助/建议都会非常有益。 谢谢。

Answer 1

以下设置对我有用：

卡夫卡 jaas 文件：

KafkaServer {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin-secret"
  user_admin="admin-secret";
};

Client {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin-secret";
};

Zookeeper jaas 文件：

Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_admin="admin-secret";
};

Kafka生产者/消费者客户端属性：

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";

Answer 2

我注意到您正在使用 Bitnami Kafka 容器运行 Kafka。 它可以使用 Docker Compose 成功运行。

首先，创建一个 docker-compose.yml 文件，如下所示：

version: '3'

services:
  zookeeper:
    image: 'bitnami/zookeeper:3.6'
    ports:
      - '2181:2181'
    environment:
      - ZOO_ENABLE_AUTH=yes
      - ZOO_SERVER_USERS=kafka
      - ZOO_SERVER_PASSWORDS=secret
      - ZOO_CLIENT_USER=kafka
      - ZOO_CLIENT_PASSWORD=secret
  kafka:
    image: 'bitnami/kafka:2.8.1'
    ports:
      - '9093:9093'
    environment:
      - ALLOW_PLAINTEXT_LISTENER=no
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_LISTENERS=INTERNAL://:9092,CLIENT://:9093,
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,CLIENT://localhost:9093
      - KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:SASL_PLAINTEXT,CLIENT:SASL_PLAINTEXT
      - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
      #Client credentials
      - KAFKA_CLIENT_USERS=user_kafka
      - KAFKA_CLIENT_PASSWORDS=secret
      #Interbroker credentials
      - KAFKA_INTER_BROKER_USER=kafka
      - KAFKA_INTER_BROKER_PASSWORD=CMwvKfeVJociGkSToMZQ
      #Zookeeper credentials
      - KAFKA_ZOOKEEPER_PROTOCOL=SASL
      - KAFKA_ZOOKEEPER_USER=kafka
      - KAFKA_ZOOKEEPER_PASSWORD=secret
    depends_on:
      - zookeeper

然后使用以下命令启动容器：

$ docker-compose up -d

# list the containers
$ docker-compose ps

最后你会发现 kafka 和 zookeeper 容器正在运行。