使用分区在 kafka -.81 中创建/更新主题

Question

I would like to create a topic in kafka using version 0.81 and do not have the kafka-topics.sh script at my disposal to do the command line create command. I need to ensure the topic has multiple partitions in order for multithreaded consumers to consume messages from the topic. Each message should only be delivered once and I was told this cannot be done without partitions in a queue even if using the same group ID.

My other question is whether a single consumer run as two separate processes can do the same thing using the same group ID? Basically I am currently not reading the messages from the queue fast enough.

Because of the older version I am using I cannot update the topic with partition count as suggested by this command:

./bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic 
   my-topic --replication-factor 1 --partitions 2

Answer 1

My other question is whether a single consumer run as two separate processes can do the same thing using the same group ID? Basically I am currently not reading the messages from the queue fast enough.

If I understand your question, the answer is YES. I mean, group.id works in the following way.

1. If it is the same, it will make, that consumers within group.id will work as with load balancer
2. If group.id is different, every group will got own "copy" of the message, so you can perform different operations on the same message content.

Because of the older version I am using I cannot update the topic with partition count as suggested by this command

I am pretty confused what are you doing here, but to change parameters of the topic, you shall use "--alter" opion/command and not "--create".