通过 kafka-python 库检查 python 中是否存在 kafka 主题，不使用消费者和 shell 命令

Question

[This is not a duplicate question because even though similar questions available on stack overflow but no answer is working in my case. Hence, I am asking my doubt here.]

I want to fetch list of available Kafka topics and check whether a particular topic exist or not. I wanna do this without using consumer and any shell command. I am looking for the solutions via libraries itself. So, to fetch list of kafka topic I did following -

import kafka
client = kafka.KafkaClient(bootstrap_servers='localhost:9092')
topicList = client.topic_partitions

But, here I am getting error as -

'KafkaClient' object has no attribute 'topic_partitions'

It would be a great help if anyone can tell me how to fix the error or suggest me any other solution. Thanks in advance.

Answer 1

You can't do this without creating a KafkaConsumer client. You get topic partitions in kafka only after they are assigned to a particular consumer.

You would have to use the KafkaConsumer client to retrieve the topics.

Other way to do this is with confluent-kafka consumer client or the admin client, though the latter retuns metadata about the cluster including the topic paritions. With admin client, you can do something like this:

from confluent_kafka.admin import AdminClient
admin_client = AdminClient({'bootstrap.servers': 'localhost:9002'})
admin_client.list_topics().topics

gives

{'topic1': TopicMetadata(topic1, $N partitions),}