I am working on a Kafka Streams application built with Spring Cloud Stream. In this application I need to:
I have considered implementing this as follows:
InteractiveQueryService
bean. The problem I hit is that the InteractiveQueryService
provides read-only access to the state store, so I cannot remove entries in the other thread. I have decided not to use Kafka Stream's punctuate capability since the semantics are different; my scheduling thread must always run at a regular interval, irrespective of the processing of the inbound messages.
Another alternative might be to use the low-level Processor API, and pass a reference to the writable state store to my scheduler thread. I will need to synchronize on write operations. But I'm not sure if this is do-able or if there are other constraints when accessing the state store like this from a separate thread.
Any input or advice would be appreciated!
my scheduling thread must always run at a regular interval, irrespective of the processing of the inbound messages
Well, punctuation based on WALL_CLOCK_TIME
does exactly what you discribed above.
The problem I hit is that the InteractiveQueryService provides read-only access to the state store
Using the Processor API and Punctuation allows you to access the state stores within the init()
with ProcessorContext#getStateStore()
and remove entries from the stores in ProcessorContext#schedule()
. The advantage of this solution is, that the processor and punctuator run in the same thread and you don't need any synchronisation between them.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.