如何在 Java 中以编程方式获取 Kafka Cluster 和 Broker 信息？

Question

I wish to get all active brokers on my kafka cluster programmatically from the worker itself. The idea is to create a health scheduler that will check and return the active brokers address.

In each worker, I set up the cluster address when I set up the consumer configuration

props.put(onsumerConfig.BOOTSTRAP_SERVERS_CONFIG, myServerAddress);

However, this will not tell me which brokers are active at the moment.

I did some searching, but I could not find any way of getting this info. Is it possible?

Answer 1

You can retrieve all details of Kafka brokers in a cluster using describeCluster() of AdminClient :

Get information about the nodes in the cluster.

Parameters: options - The options to use when getting information about the cluster.

Returns: The DescribeClusterResult .

// Create AdminClient
Properties props = new Properties();
props.load(new FileInputStream("ac.properties"));
AdminClient adminClient = KafkaAdminClient.create(props);

// Get brokers' details 
DescribeClusterResult describeClusterResult = adminClient.describeCluster();
List<Node> brokers = new ArrayList<>(describeClusterResult.nodes().get());
for (Node broker : brokers) {
    System.out.println("Host=" + broker.host() + ", Port=" + broker.port());
}