[英]Kafka Error For Topics I'm Not Subscribed To
I saw a kafka warning for topics my current instance is not subscribed to. 我看到有关当前实例未订阅的主题的kafka警告。 We use the same kafka broker, and each dev has his/her own topic. 我们使用相同的kafka经纪人,每个开发人员都有自己的主题。 However we all use the same default groupID/clientid specified in a properties file. 但是,我们都使用在属性文件中指定的相同的默认groupID / clientid。
2017-06-27T11:48:35.20-0400 [APP/PROC/WEB/0]OUT 2017-06-27 15:48:35,195 WARN [NetworkClient] - [kafka-coordinator-heartbeat-thread | ingestion-matching-kafka-consumer-group] - Error while fetching metadata with correlation id 154274 : {to-process-yi=UNKNOWN_TOPIC_OR_PARTITION, to-process-shawn=UNKNOWN_TOPIC_OR_PARTITION}
2017-06-27T11:48:35.34-0400 [APP/PROC/WEB/0]OUT 2017-06-27 15:48:35,339 INFO [PdfXmlMatcherTask] - [scheduler-1] -
2017-06-27T11:48:35.34-0400 [APP/PROC/WEB/0]OUT 2017-06-27 15:48:35,340 INFO [PdfXmlMatcherTask] - [scheduler-1] - Checking messages...
2017-06-27T11:48:35.41-0400 [APP/PROC/WEB/0]OUT 2017-06-27 15:48:35,412 WARN [NetworkClient] - [scheduler-1] - Error while fetching metadata with correlation id 154275 : {to-process-yi=UNKNOWN_TOPIC_OR_PARTITION, to-process-shawn=UNKNOWN_TOPIC_OR_PARTITION}
Is this expected? 这是预期的吗? Do I need separate group/client IDs even though each person has their own topic? 即使每个人都有自己的主题,我是否需要单独的组/客户ID?
Secondly I'm not quite sure what this error means: the heart-beat coordinator thread couldn't fetch metadata for 2 topics, but these topics exist, it's just that the consumers for those topics aren't running. 其次,我不太确定此错误的含义:心跳协调器线程无法获取2个主题的元数据,但是这些主题存在,只是这些主题的使用者未运行。
client.id - is useful for tracing requests, should be unique to each client (though not mandatory). client.id-用于跟踪请求,对于每个客户端应该唯一(尽管不是强制性的)。
groupid - this applies only to consumers. groupid-仅适用于消费者。 By your short description you probably want each consumer in its own group. 通过简短的描述,您可能希望每个消费者都属于自己的组。
I suggest you read what ConsumerGroups are for : "Kafka scales topic consumption by distributing partitions among a consumer group, which is a set of consumers sharing a common group identifier. " https://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0-9-consumer-client/ 我建议您阅读ConsumerGroups的用途:“ Kafka通过在消费者组之间分配分区来扩展主题的使用,消费者组是一组共享公用组标识符的消费者。” https://www.confluent.io/blog/tutorial-getting开始使用新的Apache Kafka-0-9消费者客户端/
UNKNOWN_TOPIC_OR_PARTITION is an error that the broker returns exactly when a topic or partition does not exist. UNKNOWN_TOPIC_OR_PARTITION是当主题或分区不存在时,代理完全返回的错误。 it's an erro that the Kafka client treats as retriable. Kafka客户将其视为可重试是一种错误。
If you think that the topic exists, please check twice. 如果您认为该主题存在,请检查两次。 It could also be the case that the topic exists but a producer or consumer is targeting directly a non-existing partition of an existing topic. 也可能是该主题存在,但生产者或消费者直接将现有主题的不存在分区作为目标。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.