設置kafka主題過濾器

Question

我正在使用本教程來設置項目。 一切正常，直到我在接收器上添加客戶容器工廠。 這是我的KafkaReciverConfig 。

@EnableKafka
@Configuration
public class ReceiverConfig {
    @Value("${spring.kafka.bootstrap-servers}")
    private String bootstrapServers;

    @Bean
    public Map<String, Object> consumerConfigs() {
        Map<String, Object> props = new HashMap<>();
        props.put(BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        props.put(GROUP_ID_CONFIG, "app.topic");
        props.put(AUTO_OFFSET_RESET_CONFIG, "earliest");
        props.put(KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
        return props;
    }

    @Bean
    public ConsumerFactory<String, TopicPayload> consumerFactory() {
        return new DefaultKafkaConsumerFactory<>(consumerConfigs(), new StringDeserializer(),
                new JsonDeserializer<>(TopicPayload.class));
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, TopicPayload> filterKafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, TopicPayload> factory = new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }
}

這是我的Receiver.liste方法

@KafkaListener(id = "app.topic", topics = "${app.topic.topicname}", containerFactory = "filterKafkaListenerContainerFactory")
public void listen(@Payload TopicPayload payload) {
    LOGGER.info("-------------------- " + payload);
}

如果我不指定containerFactory它將正常工作。 但是當我指定（我打算在這里添加一些過濾邏輯）的那一刻，我得到以下錯誤

***************************
APPLICATION FAILED TO START
***************************

Description:

Parameter 1 of method kafkaListenerContainerFactory in org.springframework.boot.autoconfigure.kafka.KafkaAnnotationDrivenConfiguration required a bean of type 'org.springframework.kafka.core.ConsumerFactory' that could not be found.
    - Bean method 'kafkaConsumerFactory' in 'KafkaAutoConfiguration' not loaded because @ConditionalOnMissingBean (types: org.springframework.kafka.core.ConsumerFactory; SearchStrategy: all) found beans of type 'org.springframework.kafka.core.ConsumerFactory' consumerFactory


Action:

Answer 1

此錯誤是由KafkaAnnotationDrivenConfiguration類引起的。

在KafkaAnnotationDrivenConfiguration類中，如果沒有名為“ kafkaListenerContainerFactory”的bean，則將kafkaListenerContainerFactory注冊為bean。

以下是在KafkaAnnotationDrivenConfiguration中注冊kafkaListenerContainerFactory bean的方法代碼。

@Bean
@ConditionalOnMissingBean(name = "kafkaListenerContainerFactory")
public ConcurrentKafkaListenerContainerFactory<?, ?> kafkaListenerContainerFactory(
        ConcurrentKafkaListenerContainerFactoryConfigurer configurer,
        ConsumerFactory<Object, Object> kafkaConsumerFactory) {
    ConcurrentKafkaListenerContainerFactory<Object, Object> factory = new ConcurrentKafkaListenerContainerFactory<>();
    configurer.configure(factory, kafkaConsumerFactory);
    return factory;
}

此方法將ConsumerFactory <Object, Object>作為參數。 但是，您注冊的ConsumerFactory是ConsumerFactory <String, TopicPayload>因此Spring找不到類型為ConsumerFactory <Object, Object>的bean並引發異常

解決方案是不要將ConsumerFactory和consumerConfigs注冊為bean。 在這種情況下，ConsumerFactory將由KafkaAutoConfiguration自動注冊為Bean。（只有在沒有類型為ConsumerFactory的bean時，KafkaAutoConfiguration才會自動注冊該bean。）

public Map<String, Object> consumerConfigs() {
    Map<String, Object> props = new HashMap<>();
    props.put(BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    props.put(GROUP_ID_CONFIG, "app.topic");
    props.put(AUTO_OFFSET_RESET_CONFIG, "earliest");
    props.put(KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
    return props;
}

public ConsumerFactory<String, TopicPayload> consumerFactory() {
    return new DefaultKafkaConsumerFactory<>(consumerConfigs(), new StringDeserializer(),
            new JsonDeserializer<>(TopicPayload.class));
}

@Bean
public ConcurrentKafkaListenerContainerFactory<String, TopicPayload> filterKafkaListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<String, TopicPayload> factory = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory());
    return factory;
}

比較簡單的方法是使用名稱kafkaListenerContainerFactory代替名稱filterKafkaListenerContainerFactory。

@Bean
public Map<String, Object> consumerConfigs() {
    Map<String, Object> props = new HashMap<>();
    props.put(BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    props.put(GROUP_ID_CONFIG, "app.topic");
    props.put(AUTO_OFFSET_RESET_CONFIG, "earliest");
    props.put(KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
    return props;
}

@Bean
public ConsumerFactory<String, TopicPayload> consumerFactory() {
    return new DefaultKafkaConsumerFactory<>(consumerConfigs(), new StringDeserializer(),
            new JsonDeserializer<>(TopicPayload.class));
}

@Bean
public ConcurrentKafkaListenerContainerFactory<String, TopicPayload> kafkaListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<String, TopicPayload> factory = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory());
    return factory;
}

有點難以解釋。 如果您知道Spring Boot的自動配置，將很容易理解。

我希望我的樣本可以。