[英]Kafka-python get number of partitions for topic
I'm using: https://github.com/mumrah/kafka-python as a kafka api in Python.我正在使用: https://github.com/mumrah/kafka-python作为 Python 中的卡夫卡 api。 I want to get the number of partitions for a specified topic.
我想获取指定主题的分区数。 How do I do that?
我怎么做?
Might be a slightly simplistic solution, but: 可能是一个稍微简单的解决方案,但是:
from kafka import KafkaClient
client = KafkaClient('SERVER:PORT')
topic_partition_ids = client.get_partition_ids_for_topic(b'TOPIC')
len(topic_partition_ids)
tested on Python 3.4.3 / kafka-python 0.9.3 在Python 3.4.3 / kafka-python 0.9.3上测试
I found this question while trying to solve this exact same problem. 我试图解决这个完全相同的问题时发现了这个问题。 I know the question is old, but here is the solution I came up with (using Kazoo to talk to zookeeper):
我知道问题是陈旧的,但这是我提出的解决方案(使用Kazoo与zookeeper交谈):
from kazoo.client import KazooClient
class KafkaInfo(object):
def __init__(self, hosts):
self.zk = KazooClient(hosts)
self.zk.start()
def topics(self):
return self.zk.get_children('/brokers/topics')
def partitions(self, topic):
strs = self.zk.get_children('/brokers/topics/%s/partitions' % topic)
return map(int, strs)
def consumers(self):
return self.zk.get_children('/consumers')
def topics_for_consumer(self, consumer):
return self.zk.get_children('/consumers/%s/offsets' % consumer)
def offset(self, topic, consumer, partition):
(n, _) = self.zk.get('/consumers/%s/offsets/%s/%d' % (consumer, topic, partition))
return int(n)
Python 3.8.10/kafka-python 2.0.2 Solution: Python 3.8.10/kafka-python 2.0.2 解决方案:
from kafka import KafkaConsumer
def get_partitions_number(server, topic):
consumer = KafkaConsumer(
topic,
bootstrap_servers=server
)
partitions = consumer.partitions_for_topic(topic)
return len(partitions)
For those of you using Confluent-Python or the enterprise API. 对于那些使用Confluent-Python或企业API的人。 This can be done this way:
这可以这样做:
def count_partitions(my_partitions) -> int:
count = 0
for part in my_partitions:
count = count + 1
return count
cluster_data: ClusterMetadata = producer.list_topics(topic=TOPIC)
topic_data: TopicMetadata = cluster_data.topics[TOPIC]
available_partitions: PartitionMetadata = topic_data.partitions
print(count_partitions(available_partitions))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.