How to create topics if it does not exists in Kafka dynamically using kafka-python

Question

I'm fairly new to Python and just getting started with Kafka. I am using the library named python-kafka to communicate with Kafka. Now I have a requirement that I need to create topics dynamically, however if it does exists, I do not need to create it.

From reading the docs I see that I can use KafkaAdminClient to create and delete topics, however I do not find any to check if topic exists.

Answer 1

The KafkaAdminClient does not expose a method to list topics but you can get the list of existing topics by simply querying the cluster metadata from a KafkaClient .

For example, this will print all the topics in the cluster:

from kafka.client import KafkaClient

client = KafkaClient(bootstrap_servers='localhost:9092')

future = client.cluster.request_update()
client.poll(future=future)

metadata = client.cluster
print(metadata.topics())