Kafka producer/consuper Topic not authorized

Question

Whenever I try to connect to kafka to producer/consume I get "Not authorized for topics [test2]"

If I turn off the authorization I get authenticated successfully, so the authentication works and only the authorization doesn't.

ACL authorization with kafka.security.auth.SimpleAclAuthorizer not working.

config/server.properties

authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
listeners=SASL_PLAINTEXT://kafka3:9092
security.inter.broker.protocol= SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
delete.topic.enable=false

logs/kafka-authorizer

[2019-04-06 13:24:05,693] DEBUG No acl found for resource Topic:LITERAL:test2, authorized = false (kafka.authorizer.logger) [2019-04-06 13:24:05,695] INFO Principal = User:alice is Denied Operation = Describe from host = 10.0.9.20 on resource = Topic:LITERAL:test2(kafka.authorizer.logger)uper.users=User:admin

server's jaas file:

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin"
    user_admin="admin"
    user_alice="alice";
};

bin/kafka-server-start.sh

$base_dir/kafka-run-class.sh $EXTRA_ARGS -Djava.security.auth.login.config=$base_dir/../config/jaas-kafka-server.conf kafka.Kafka "$@"

acl output:

Current ACLs for resource `Topic:LITERAL:test2`:
    User:alice has Allow permission for operations: Write from hosts: *

Answer 1

User alice is currently only authorized to Write to that topic. You would also likely want to add the ACLs Describe and Read to be able to properly produce and consume to your existing topic.

The kafka-acls tool provides convenience options --consumer and --producer when adding ACLs to a topic. Otherwise, you can use --operation to add specific operations such as Describe . By adding Describe , you'd remove the log you're currently seeing in logs/kafka-authorizer .

Answer 2

As you would like consume and produce message from a particular topic with turning on ACL, then you need to apply ACL on the topic to consume and produce message to it. You need to do it through a super user like kafka.

Login to a kafka broker then use below command :

sudo su - kafka

kinit -kt /path/to/keytabs/kafka.service.keytab kafka/serviceprincipal name@domain name (you can get it from kafka jaas file)

Then from kafka directory execute these command :

bin/kafka-acls --add --allow-principal User:* --consumer --topic test2 --authorizer-properties zookeeper.connect=:2181 --group *

Similarly for producer to push message to topics :

bin/kafka-acls.sh --add --allow-principal User:* --producer --topic test2 --authorizer-properties zookeeper.connect=:2181

Above command will apply ACLs to all users . You can restrict it by specifying individual user name instead of '*' in the command.

** Remember you should not have any text file or any other file other than kafka installed file/directory in the bin directory as part of kafka installation.

To get more info on ACL(addind/removing,listing) go to below link:

https://docs.confluent.io/current/kafka/authorization.html