简体   繁体   English

如何从特定的偏移量中消费一个kafka主题?

[英]how to consume a kafka topic from a specific offset?

recently I am using kafka, I have a topic and I am using the following code to consume最近我在使用kafka,我有一个主题,我正在使用以下代码来消费

@KafkaListener(topics = "topic_name", groupId =  "_id" , id = "pro", containerFactory = "kafkaListenerContainerFactory") 
    public void consume(ConsumerRecord<String, String> record, Acknowledgment ack) {            
        kafkaService.proccessorConsumer(record);
        ack.acknowledge();

    }

every thing works fine, but I need to handle a situation where if the service stopped for any reason, then started I want to continue consuming from the last message that has processed, I do understand that the acknowledgment help with this, but for the sake of certainty I saved the last consumed offset somewhere.一切正常,但我需要处理这样一种情况,如果服务因任何原因停止,然后开始我想继续使用已处理的最后一条消息,我明白确认对此有帮助,但为了方便可以肯定的是,我在某处保存了最后消耗的偏移量。 my question is how I could use that offset to start consuming the topic from it.我的问题是如何使用该偏移量开始使用它的主题。

As @OneCricketeer indicates, what you are trying to achieve is the default behaviour of the Kafka consumer, if you haven't disabled automatic commit.正如@OneCricketeer 所指出的,如果您没有禁用自动提交,您想要实现的是 Kafka 消费者的默认行为。

You can check this by describing your consumer group using the consumer id as follows, just check that the offset of your consumer is the same as the one you have stored elsewhere.您可以通过如下使用消费者 ID 描述您的消费者组来检查这一点,只需检查您的消费者的偏移量是否与您在其他地方存储的偏移量相同。

> bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group-id

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何在特定偏移量到特定偏移量中使用来自 kafka 主题的数据? - How to consume data from kafka topic in specific offset to specific offset? 如何在 Java 中使用来自 Kafka 的消息,从特定偏移量开始 - How to consume messages from Kafka in Java, starting from a specific offset 从 Kafka 主题中消费字符串 - Consume String from Kafka Topic 从Apache Camel中的特定Offset开始阅读Kafka主题 - Start reading Kafka topic from specific Offset in Apache Camel 如何使用一些过滤器使用来自 Kafka 主题的消息? - How to consume message from Kafka topic using some filter? 如何使用 Java Spark 结构化流从 Kafka 主题正确消费 - How to consume correctly from Kafka topic with Java Spark structured streaming 如何使用 spring webflux 从 Kafka 主题中持续消费? - How to continually consume from Kafka topic using spring webflux? 存在偏移量时如何从最大偏移量中消耗kafka? - How to consume kafka frome the max offset when the offset is exist? 如何使用kafka客户端以便使用Java语言从主题中进行消费 - How to use kafka client in order to consume from topic using java language 是否可以分别使用来自 kafka 主题的每条消息? - Is it posible to consume each message from a kafka topic separately?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM