kafka metrics missing in micrometer since SpringBoot 2.3.x

Question

Upon switching from SpringBoot 2.2.13 to any of 2.3.x - 2.6.x I no longer have kafka metrics published in micrometer.

I am not using spring-kafka, just manually creating KafkaStreams or KafkaConsumer, but that worked with 2.2.x. Adding the spring-kafka dependency doesn't help. Does anyone know what I have to do to get these back?

Answer 1

I think if you are manually creating those components you should manually register them, see KafkaMetrics (I'm not sure how this worked in 2.2.x). You can also check how Spring Boot does this in KafkaMetricsAutoConfiguration .

Answer 2

Since Spring Boot 2.3 Kafka metrics names have been changed.

kafka_consumer_records_consumed_total_records_total -> kafka_consumer_fetch_manager_records_consumed_total
kafka_consumer_records_lag_records -> kafka_consumer_fetch_manager_records_lag
kafka_consumer_records_lag_max_records -> kafka_consumer_fetch_manager_records_lag_max
kafka_consumer_fetch_latency_max_seconds -> kafka_consumer_fetch_manager_fetch_latency_max
kafka_consumer_bytes_consumed_total_bytes_total -> kafka_consumer_fetch_manager_bytes_consumed_total
kafka_consumer_records_per_request_avg_records -> kafka_consumer_fetch_manager_records_per_request_avg
kafka_consumer_heartbeat_rate_heartbeats -> kafka_consumer_coordinator_heartbeat_rate

If you have custom consumer factories, add customizers.orderedStream().forEach(customizer -> customizer.customize(consumerFactory));

@Bean
public ConsumerFactory<Integer, String> customConsumerFactory(ObjectProvider<DefaultKafkaConsumerFactoryCustomizer> customizers) {
    DefaultKafkaConsumerFactory<Integer, String> consumerFactory = new DefaultKafkaConsumerFactory<>(customConsumerConfigs());
    customizers.orderedStream().forEach(customizer -> customizer.customize(consumerFactory));
    return consumerFactory;
}