Kafka Streams: InvalidStateStoreException

Question

If the stateful stream application is started with 6 threads on a single node, would the above exception occur?

Is there any process that needs to be followed, if a stateful stream application started on node 1 consuming a particular topic, is made to run on different node?

If the stateful stream application is started on 2 nodes and if the above exception occurs, would the stream application terminate immediately?

• If yes, where can this exception be caught in a try-catch block?
  If the exception can be caught, and if we add sleep for 10 mins, would the store automatically gets to valid state?
• If not, is there a method that can be used to check the store state and wait until it becomes valid?

---

Follow-up:

If the stateful stream application is started with 6 threads on a single node, would the above exception occur?

It can

Essentially I was wondering if we keep the entire topic consumption on a single node, would it avoid re-building the store from an internal topic if a re-balancing occurs, due to one of the thread going down/terminates?

store is not ready yet: you can wait until the store is ready -- best to register a restore callback (check the docs for details) to get informed when restore is finished and you can retry to query the store.

Sorry, just to be clear on the above, is it StateRestoreCallback OR StateRestoreListener? I assume it is the later one. Also, is it required to override StateRestoreCallback and include logic to restore the store?

Answer 1

InvalidStateStoreException can have different causes, thus, it's hard to answer your question without more context.

If the stateful stream application is started with 6 threads on a single node, would the above exception occur?

It can.

Is there any process that needs to be followed, if a stateful stream application started on node 1 consuming a particular topic, is made to run on different node?

No.

If the stateful stream application is started on 2 nodes and if the above exception occurs, would the stream application terminate immediately?

Depends where the exception it thrown:

• Either, the corresponding StreamThread would die, but the application would not terminate automatically. You should register an uncaught exception handler on the KafkaStreams instance and react to an dying thread with custom code (like, terminating the application).
• If it is thrown from KafkaStreams using interactive queries, StreamThread would not be affected.

Where can this exception be caught in a try-catch block?

Usually yes, especially if you refer to interactive queries feature.

if we add sleep for 10 mins, would the store automatically gets to valid state?

If you refer to interactive queries feature, sleeping is not a good strategy. There are multiple causes for the exception and you need to react accordingly:

• store is not local but on different node: you can figure this out by check the store metadata.
• store is not ready yet: you can wait until the store is ready -- best to register a restore listener (check the docs for details) to get informed when restore is finished and you can retry to query the store.

Update

Essentially I was wondering if we keep the entire topic consumption on a single node, would it avoid re-building the store from an internal topic if a re-balancing occurs, due to one of the thread going down/terminates?

Yes (for non-EOS case). Other threads would detect the local store and reuse it.

StateRestoreCallback OR StateRestoreListener

Yes, it's StateRestoreListener . You would implement StateRestoreCallback only if you write a custom state store.