Why is my spring cloud stream config creating multiple kafka consumers

Question

It's not a huge problem but I'm curious where some extra stream consumers are coming from, and if that's a setting I can change.

I've got a very simple spring cloud stream consumer setup against a local Kafka broker. Here's the spring config

spring:
  cloud:
    stream:
      bindings:
        consumer-in-0:
          destination: test-topic
          group: test-group

And the consumer class itself:

@Bean
Consumer<Message<String>> consumer() {
 return message -> System.out.println("Got it: " + message.getPayload());
}

When I run the app though, I can see 3 consumers created in the output. But when I check the consumer-group members in my local broker, it's always just one consumer, and it's always the second consumer created (ie with client id test-group-2 )

Just for clarity, I'm using Spring Boot version 2.3.4.RELEASE and cloud dependencies version Hoxton.SR10 .

And here's the dependencies in the pom:

<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-devtools</artifactId>
    <scope>runtime</scope>
    <optional>true</optional>
  </dependency>
  <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
  </dependency>
<dependencies>

Why am I getting 3 consumers? Why is the second one the only one that actually listens on the Kafka topic?

Answer 1

During start up, a temporary consumer is created to get information about the partitions provisioned for the topic.

The second consumer is the real consumer.

If you have the actuator (actually Micrometer) on the classpath the KafkaBinderMetrics creates another consumer so it can calculate the lag. It does not actually consume anything.