Purpose of Zookeeper in Kafka

Question

As from the latest consumer versions of Kafka, the consumers aren't dependent on ZooKeeper. But " https://kafka.apache.org/ " says Kafka requires Zookeeper, so start zookeeper server. why is it so?. Once a topic has been created, even though I terminate Zookeeper it works. So the purpose of Zookeeper is only for creating a Topic? If so why not move creating Topic also to be independent of zookeeper

Answer 1

Kafka topics still require Zookeeper for electing a leader, communicating server failure, and storing the list of topics, plus some extra metadata such as replica location and topic configurations.

From what I've heard, Confluent and the Kafka community are trying to move away from the Zookeeper dependency. For example, the Avro Schema Registry can now use Kafka for leader election. And in Confluent Cloud, you have no access to Zookeeper at all.

Kafka Wiki - How does Kafka depend on Zookeeper

Answer 2

The consumers are not dependent on Zookeeper as they are client-side. Likewise with the producers.

Zookeeper is required for the Kafka brokers themselves. Kafka brokers use Zookeeper to co-ordinate and synchronise themselves.