[英]Unable to open store for Kafka streams because invalid state
I am trying to work with Kafka Streams and I have created the following Topology:我正在尝试使用 Kafka Streams,并且创建了以下拓扑:
KStream<String, HistoryEvent> eventStream = builder.stream(applicationTopicName, Consumed.with(Serdes.String(),
historyEventSerde));
eventStream.selectKey((key, value) -> new HistoryEventKey(key, value.getIdentifier()))
.groupByKey()
.reduce((e1, e2) -> e2, Materialized.as(streamByKeyStoreName));
I later start the streams like this:我后来开始这样的流:
private void startKafkaStreams(KafkaStreams streams) {
CompletableFuture<KafkaStreams.State> stateFuture = new CompletableFuture<>();
streams.setStateListener((newState, oldState) -> {
if(stateFuture.isDone()) {
return;
}
if(newState == KafkaStreams.State.RUNNING || newState == KafkaStreams.State.ERROR) {
stateFuture.complete(newState);
}
});
streams.start();
try {
KafkaStreams.State finalState = stateFuture.get();
if(finalState != KafkaStreams.State.RUNNING) {
// ...
}
} catch (InterruptedException ex) {
// ...
} catch(ExecutionException ex) {
// ...
}
}
My Streams start without an error and they eventually get into the state of RUNNING
where the future is completed.我的 Streams 开始时没有错误,它们最终进入
RUNNING
状态,在那里完成未来。 Later I am trying to access that store that I created in my topology for the KTable:后来我尝试访问我在我的拓扑中为 KTable 创建的存储:
public KafkaFlowHistory createFlowHistory(String flowId) {
ReadOnlyKeyValueStore<HistoryEventKey, HistoryEvent> store = streams.store(streamByKeyStoreName,
QueryableStoreTypes.keyValueStore());
return new KafkaFlowHistory(flowId, store, event -> topicProducer.send(new ProducerRecord<>(applicationTopicName, flowId, event)));
}
I have verified that the createFlowHistory
is called after the initializing future is completed in RUNNING
state, however I am consistently unable to do this and KafkaStreams is reporting the following error:我已经验证在
RUNNING
状态下完成初始化未来后调用createFlowHistory
,但是我始终无法执行此操作并且 KafkaStreams 报告以下错误:
Exception in thread "main" org.apache.kafka.streams.errors.InvalidStateStoreException: Cannot get state store flow-event-stream-file-service-test-instance-by-key because the stream thread is PARTITIONS_ASSIGNED, not RUNNING
线程“main” org.apache.kafka.streams.errors.InvalidStateStoreException 中的异常:无法获取状态存储流-事件-流-文件-service-test-instance-by-key 因为流线程是 PARTITIONS_ASSIGNED,而不是 RUNNING
Apparently the state of the thread has changed.显然线程的状态已经改变。 Do I need to take care of this manually when trying to query a store and wait for the internal thread of Kafka to get into the right state?
在尝试查询商店并等待 Kafka 的内部线程进入正确状态时,我是否需要手动处理这个问题?
Older Versions ( before 2.2.0)旧版本(2.2.0之前)
On startup, Kafka Streams does the following state transitions:在启动时,Kafka Streams 执行以下状态转换:
CREATED -> RUNNING -> REBALANCING -> RUNNING
You need to wait for the second RUNNING state before you can query.您需要等待第二个 RUNNING 状态才能查询。
New Version: as of 2.2.0新版本:从 2.2.0 开始
The state transition behavior on startup was changed (via https://issues.apache.org/jira/browse/KAFKA-7657 ) to:启动时的状态转换行为(通过https://issues.apache.org/jira/browse/KAFKA-7657 )更改为:
CREATED -> REBALANCING -> RUNNING
Hence, you should not hit this issue any longer.因此,您不应再遇到此问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.