Kafka zookeeper authentication not working

Question

I am trying to enable SASL username and password for a Kafka cluster with no ssl. I followed the steps on this Stackoverflow:

Kafka SASL zookeeper authentication

SASL authentication seems to be working for Kafka brokers. consumers and producers have to authenticate before writing to or reading from a topic. So far so good.

The problem is with creating and deleting topics on kafka. when I try to use the following command for example:

~/kafka/bin/kafka-topics.sh --list --zookeeper 10.x.y.z:2181

I am able to list all topics in the kafka cluster and create or delete any topic with no authentication at all.

I tried to follow the steps here:

Super User Authentication and Authorization

but nothing seem to work.

Any help in this matter is really appreciated.

Thanks & Regards, Firas Khasawneh

Answer 1

You need to add zookeeper.set.acl=true to your Kafka server.properties so that Kafka will create everything in zookeeper with ACL set. For the topics which are already there, there will be no ACL and everyone can remove them directly from zookeeper.

Actually because of that mess, I had to delete everything from my zookeeper and Kafka and start from scratch.

But once everything is set, you can open zookeeper shell to verify that the ACL is indeed set:

KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/your/jaas.conf" bin/zookeeper-shell.sh XXXXX:2181

From the shell you can run: getAcl /brokers/topics and check that not anyone from world have cdrwa

On a side note, the link you provided doesn't seem to reflect how the current version of Kafka stores information in zookeeper. I briefly looked at the codes and for those kafka-topics.sh commands, the topics information is from /brokers/topics instead of /config/topics