[英]Cannot query local state store in Kafka Streams Application
我正在用spring-kafka构建一个kafka流应用程序,以按键对记录进行分组并应用一些业务逻辑。 我正在遵循spring-kafka-streams doc上所述的配置,但是问题是当我想从本地存储中检索值时,出现以下错误:
org.apache.kafka.streams.errors.InvalidStateStoreException: The state store, user-data-response-count, may have migrated to another instance.
at org.apache.kafka.streams.state.internals.QueryableStoreProvider.getStore(QueryableStoreProvider.java:60)
at org.apache.kafka.streams.KafkaStreams.store(KafkaStreams.java:1053)
at com.umantis.management.service.UserDataManagementService.broadcastUserDataRequest(UserDataManagementService.java:121)
这是我的KafkaStreamsConfiguration:
@Configuration
@EnableConfigurationProperties(EventsKafkaProperties.class)
@EnableKafka
@EnableKafkaStreams
public class KafkaConfiguration {
@Value("${app.kafka.streams.application-id}")
private String applicationId;
// This contains both the bootstrap servers and the schema registry url
@Autowired
private EventsKafkaProperties eventsKafkaProperties;
@Bean(name = KafkaStreamsDefaultConfiguration.DEFAULT_STREAMS_CONFIG_BEAN_NAME)
public StreamsConfig streamsConfig() {
Map<String, Object> props = new HashMap<>();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, applicationId);
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, this.eventsKafkaProperties.getBrokers());
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, SpecificAvroSerde.class);
props.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, this.eventsKafkaProperties.getSchemaRegistryUrl());
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
return new StreamsConfig(props);
}
@Bean
public KGroupedStream<String, UserDataResponse> responseKStream(StreamsBuilder streamsBuilder, TopicUtils topicUtils) {
final Map<String, String> serdeConfig = Collections.singletonMap("schema.registry.url", this.eventsKafkaProperties.getSchemaRegistryUrl());
final Serde<UserDataResponse> valueSpecificAvroSerde = new SpecificAvroSerde<>();
valueSpecificAvroSerde.configure(serdeConfig, false);
return streamsBuilder
.stream("myTopic", Consumed.with(Serdes.String(), valueSpecificAvroSerde))
.groupByKey();
}
这是我的服务代码在getKafkaStreams().store
上失败:
@Slf4j
@Service
public class UserDataManagementService {
private static final String RESPONSE_COUNT_STORE = "user-data-response-count";
@Autowired
private StreamsBuilderFactoryBean streamsBuilderFactory;
public UserDataResponse broadcastUserDataRequest() {
this.responseGroupStream.count(Materialized.as(RESPONSE_COUNT_STORE));
if (!this.streamsBuilderFactory.isRunning()) {
throw new KafkaStoreNotAvailableException();
}
// here we should have a single running kafka instance
ReadOnlyKeyValueStore<String, Long> countStore =
this.streamsBuilderFactory.getKafkaStreams().store(RESPONSE_COUNT_STORE, QueryableStoreTypes.keyValueStore());
...
}
上下文:我在春季启动测试中在单个实例上运行应用程序,并且确保kafka实例处于运行状态。 我从apache上搜索了有关此问题的文档,但是我的案例似乎不匹配。
谁能指出我做错了什么以及可能的解决方案?
我在Kafka Streams上还很新,所以任何帮助将不胜感激。
好的,只是看到我在问streams工厂是否正在运行,但是我没有问kakfa streams实例是否在实际运行。
通过对streamsBuilderFactory.getKafkaStreams().state
轮询可以解决此问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.