如何获取 Confluent.Kafka 中某个主题的所有“PartitionTopic”列表？

Question

I am using confluent-kafka-dotnet (Confluent.Kafka) to produce messages for Kafka.

I would like to manually manage which partitions messages go to in order to preserve order for certain groups of messages.

How is it possible to get a list of PartitionTopic for a Kafka topic?

I have looked at this solution in Java. But I don't understand how to achieve the same functionality with Confluent.Kafka .

Alternatively, it would be acceptable to send messages with keys, because the same keys are guaranteed to be on the same partitions. But again, I could not find a way to create a new Message with any key type other than Null . So, an example of sending a message with a non-null key would be helpful.

Answer 1

To get a list of TopicPartitions for a single topic you could use AdminClient class:

using (var adminClient = new AdminClientBuilder(new AdminClientConfig { BootstrapServers = "bootstrap-servers" }).Build())
{
    var meta = adminClient.GetMetadata(TimeSpan.FromSeconds(20));

    var topic = meta.Topics.SingleOrDefault(t => t.Topic == "topic-name");

    var topicPartitions = topic.Partitions;
}

You can find more AdminClient examples here . But notice the warnings, saying that: "The API for this functionality is subject to change."

Answer 2

I have worked out that the reason I could not create messages with a non-null key was because I had specified my Producer with <Null, ...> .

Thank you to mjwills for asking me for minimal reproducible example, which prompted me to work this out.