[英]Kaka auto.offset.reset query
My project uses Kafka 0.10.2 version.我的项目使用 Kafka 0.10.2 版本。 Iam setting enable.auto.commit=false and auto.offset.reset=latest in the consumer.
我在消费者中设置 enable.auto.commit=false 和 auto.offset.reset=latest。 If consumer is restarted after maintenance, the consumer is reading again from first offset instead of waiting for latest offset messages.
如果消费者在维护后重新启动,消费者将再次从第一个偏移量读取,而不是等待最新的偏移量消息。 Any reasons why is this happening?
发生这种情况的任何原因? Have i understood the configurations wrongly?
我是否错误地理解了配置?
My requirement is the consumer should not auto commit and should read only the new messages put into the topic when it is active.我的要求是消费者不应该自动提交,并且应该只读取主题处于活动状态时放入的新消息。
Just because you aren't auto committing doesn't guarnatee there are no manual commits.仅仅因为您没有自动提交并不能保证没有手动提交。
Regardless, auto.offset.reset=latest
will never send the consumer group to the beginning of the topic.无论如何,
auto.offset.reset=latest
永远不会将消费者组发送到主题的开头。 Sounds like whatever Kafka tool / library you are using is calling a consumer.seekToBeginning
call on its own.听起来您正在使用的任何 Kafka 工具/库都在自行调用
consumer.seekToBeginning
调用。
For Understanding purpose, The Consumer property auto.offset.reset
determines what to do if there is no valid offset in Kafka for the Consumer's Consumer Group Based on the below scenarios:出于理解目的,Consumer 属性
auto.offset.reset
根据以下场景确定如果 Kafka 中没有针对 Consumer 的 Consumer Group 的有效偏移量时要执行的操作:
– When a particular Consumer Group starts the first time – 当特定的消费者组第一次启动时
– If the Consumer offset is less than the smallest offset – 如果Consumer offset小于最小offset
– If the Consumer offset is greater than the last offset – 如果Consumer offset大于上一个offset
▪ The value can be one of: ▪ 该值可以是以下之一:
– earliest
: Automatically reset the offset to the earliest available –
earliest
:自动将偏移量重置为最早可用的
– latest
: Automatically reset to the latest offset available –
latest
:自动重置为可用的最新偏移量
– none
: Throw an exception if no previous offset can be found for the ConsumerGroup –
none
:如果找不到 ConsumerGroup 的先前偏移量,则抛出异常
▪ The default is latest
▪ 默认是
latest
的
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.