简体繁体 English

Kafka Consumer配置——auto.offset.reset如何控制消息消费

[英]Kafka Consumer configuration - How does auto.offset.reset controls the message consumption

原文 2019-11-13 01:36:06 6 1 apache-kafka

I'm trying to understand, how does the ConsumerConfig.auto.offset.reset = latest would affect the message consumption.我试图了解 ConsumerConfig.auto.offset.reset = latest 将如何影响消息消费。

For example I've a consumer, sending 100 messages initially at time t1 and then my consumer is up and running at t1+30 sec, then would my consumer consume the messages published after t1+30 sec or will it consume messages published t1 onwards?例如，我有一个消费者，最初在时间 t1 发送 100 条消息，然后我的消费者在 t1+30 秒启动并运行，然后我的消费者将使用在 t1+30 秒之后发布的消息还是会使用从 t1 开始发布的消息?

1 个解决方案

It depends.这取决于。

auto.offset.reset only applies when there is no stored offset for the consumer group. auto.offset.reset仅适用于没有为消费者组存储偏移量的情况。

It applies to the following conditions:它适用于以下条件：

the first time a consumer group consumes消费者组第一次消费
if a consumer doesn't commit any offsets, the next time it is started如果消费者没有提交任何偏移量，则在下次启动时
if a consumer group has been expired (7 days by default with modern brokers)如果消费者组已过期（现代经纪人默认为 7 天）
if the message the stored offset points to has been removed due to message retention policies (an attempt to read a message that has been purged triggers the application of the rule)如果存储的偏移量指向的消息由于消息保留策略而被删除（尝试读取已被清除的消息会触发规则的应用）

If a consumer commits an offset;如果消费者提交了偏移量； it will start at the last committed offset the next time it is started.它将在下次启动时从最后提交的偏移量开始。