简体繁体 English

Kafka流：主题分区->消费者1：1映射未发生

[英]Kafka streams: topic partitions->consumer 1:1 mapping not happening

原文 2018-08-31 02:09:00 4 1 apache-kafka/ apache-kafka-streams

I have written a streams application to talk to topic on cluster of 5 brokers with 10 partitions. 我已经编写了一个流应用程序，以讨论5个具有10个分区的代理群集的主题。 I have tried multiple combinations here like 10 application instances (on 10 different machines) with 1 stream thread each, 5 instances with 2 threads each. 我在这里尝试了多种组合，例如10个应用实例（在10台不同的计算机上），每个实例有1个流线程，5个实例有2个线程。 But for some reason, when I check in kafka manager, the 1:1 mapping between partition and stream thread is not happening. 但是由于某种原因，当我签入kafka管理器时，分区和流线程之间的1：1映射没有发生。 Some of the threads are picking up 2 partitions while some picking up none. 一些线程正在拾取2个分区，而有些则没有拾取。 Can you please help me with same?? 你能帮我吗？ All threads are part of same group and subscribed to only one topic. 所有线程都属于同一组，并且仅订阅一个主题。 The kafka streams version we are using is 0.11.0.2 and broker version is 0.10.0.2 我们使用的kafka流版本是0.11.0.2，代理版本是0.10.0.2

Thanks for your help 谢谢你的帮助

1 个解决方案

Maybe you are hitting https://issues.apache.org/jira/browse/KAFKA-7144 -- I would recommend to upgrade to the latest versions. 也许您正在访问https://issues.apache.org/jira/browse/KAFKA-7144-我建议您升级到最新版本。

Note: you do not need to upgrade your brokers 注意：您不需要升级经纪人

卡夫卡 - 主题与分区与消费者 - Kafka - Topic & Partitions & Consumer

Kafka Streams：内部主题分区 - Kafka Streams: Internal topic partitions

Kafka Streams：从具有多个分区的主题中读取 - Kafka Streams: Reading from topic with multiple partitions

Kafka 流中的更改日志主题 - 设置或更改分区 - changelog topic in Kafka streams - setting or altering partitions

Kafka 消费者获得特定主题的分配分区 - Kafka Consumer get assigned partitions for a specific topic

Kafka使用者 - 消费者进程和线程与主题分区的关系是什么 - Kafka consumer - what's the relation of consumer processes and threads with topic partitions

python3.6中如何将kafka主题的所有分区分配给消费者？ - How to assign all partitions of a kafka topic to a consumer in Python3.6?

Kafka 主题仍然与 Kafka Streams 应用程序的旧消费者组相关联 - Kafka topic is still associating with old consumer group of Kafka Streams application

Kafka 的消费者线程数应该等于主题分区数 - Kafka Should Number of Consumer Threads equal number of Topic Partitions

Consumer.poll无法从具有3个分区的Kafka Topic中获取记录 - consumer.poll not fetching records from Kafka Topic having 3 partitions

暂无

暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 卡夫卡 - 主题与分区与消费者 - Kafka - Topic & Partitions & Consumer Kafka Streams：内部主题分区 - Kafka Streams: Internal topic partitions Kafka Streams：从具有多个分区的主题中读取 - Kafka Streams: Reading from topic with multiple partitions Kafka 流中的更改日志主题 - 设置或更改分区 - changelog topic in Kafka streams - setting or altering partitions Kafka 消费者获得特定主题的分配分区 - Kafka Consumer get assigned partitions for a specific topic Kafka使用者 - 消费者进程和线程与主题分区的关系是什么 - Kafka consumer - what's the relation of consumer processes and threads with topic partitions python3.6中如何将kafka主题的所有分区分配给消费者？ - How to assign all partitions of a kafka topic to a consumer in Python3.6? Kafka 主题仍然与 Kafka Streams 应用程序的旧消费者组相关联 - Kafka topic is still associating with old consumer group of Kafka Streams application Kafka 的消费者线程数应该等于主题分区数 - Kafka Should Number of Consumer Threads equal number of Topic Partitions Consumer.poll无法从具有3个分区的Kafka Topic中获取记录 - consumer.poll not fetching records from Kafka Topic having 3 partitions

相关标签

粤ICP备18138465号 © 2020-2024 STACKOOM.COM