[英]How to consume data from kafka topic in specific offset to specific offset?
我需要消耗特定的偏移量到特定的结束偏移量!! consumer.seek() 从特定偏移量读取数据,但我需要将数据从偏移量检索到 tooffset !! 任何帮助将不胜感激,提前致谢。
ConsumerRecords<String, String> records = consumer.poll(100);
if(flag) {
consumer.seek(new TopicPartition("topic-1", 0), 90);
flag = false;
}
要从起始偏移量到结束偏移量读取消息,首先需要使用seek()
将消费者移动到所需的起始位置,然后使用poll()
直到达到所需的结束偏移量。
例如,要从偏移量 100 到 200 进行消耗:
String topic = "test";
TopicPartition tp = new TopicPartition(topic, 0);
try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(configs)) {
consumer.subscribe(Arrays.asList(topic), new ConsumerRebalanceListener() {
@Override
public void onPartitionsRevoked(Collection<TopicPartition> partitions) {}
@Override
public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
// Move to the desired start offset
consumer.seek(tp, 100L);
}
});
boolean run = true;
long lastOffset = 200L;
while (run) {
ConsumerRecords<String, String> crs = consumer.poll(Duration.ofMillis(100L));
for (ConsumerRecord<String, String> record : crs) {
System.out.println(record);
if (record.offset() == lastOffset) {
// Reached the end offsey, stop consuming
run = false;
break;
}
}
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.