无论如何对kafka流应用程序中的不同输入主题使用不同的auto.offset.reset策略？

Question

The use case is: I have a kafka streams app that consumer from an input topic, and output to a intermediate topic, then in the same streams another topology consume from this intermediate topic.

Whenever the application id is updated, both topic start to consumer from earliest. I want to change the auto.offset.reset for the intermediate topic to latest while keep that to earliest for the input topic.

Answer 1

Yes. You can set the reset strategy for each topic via:

// Processor API
topology.addSource(AutoOffsetReset offsetReset, String name, String... topics); 

// DSL
builder.stream(String topic, Consumed.with(AutoOffsetReset offsetReset));
builder.table(String topic, Consumed.with(AutoOffsetReset offsetReset));

All those methods have some overloads that allow to set it.