@StreamListener not receiving message from kafka topic

Question

I am able to send and receive the message using code:

@EnableBinding(Processor.class)
public class KafkaStreamsConfiguration {
  @StreamListener(Processor.INPUT)
  @SendTo(Processor.OUTPUT)
  public String processMessage(String message) {
    System.out.println("message = " + message);
    return message.replaceAll("my", "your");
  }
}


@RunWith(SpringRunner.class)
@SpringBootTest
@DirtiesContext
public class StreamApplicationIT {
private static String topicToPublish = "eventUpdateFromEventModel";

@BeforeClass
public static void setup() {
    System.setProperty("spring.kafka.bootstrap-servers", embeddedKafka.getBrokersAsString());
}

@Autowired
private KafkaMessageSender<String> kafkaMessageSenderToTestErrors;

@Autowired
private KafkaMessageSender<EventNotificationDto> kafkaMessageSender;

@ClassRule
public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, topicToPublish);

@Autowired
private Processor pipe;

@Autowired
private MessageCollector messageCollector;


@Rule
public OutputCapture outputCapture = new OutputCapture();

@Test
public void working() {
    pipe.input()
            .send(MessageBuilder.withPayload("This is my message")
                    .build());

    Object payload = messageCollector.forChannel(pipe.output())
            .poll()
            .getPayload();

    assertEquals("This is your message", payload.toString());
}

@Test
public void non_working() {
    kafkaMessageSenderToTestErrors.send(topicToPublish, "This was my message");
    assertTrue(isMessageReceived("This was your message", 50));
}

private boolean isMessageReceived(final String msg, final int maxAttempt) {
    return IntStream.rangeClosed(0, maxAttempt)
            .peek(a -> {
                try {
                    TimeUnit.MILLISECONDS.sleep(100);
                } catch (InterruptedException e) {
                    fail();
                }
            }).anyMatch(i -> outputCapture.toString().contains(msg));
}

}

@Service
@Slf4j
public class KafkaMessageSender<T> {
    private final KafkaTemplate<String, byte[]> kafkaTemplate;
    private final ObjectWriter objectWriter;

    public KafkaMessageSender(KafkaTemplate<String, byte[]> kafkaTemplate, ObjectMapper objectMapper) {
        this.kafkaTemplate = kafkaTemplate;
        this.objectWriter = objectMapper.writer();
    }

    public void send(String topicName, T payload) {
        try {
            kafkaTemplate.send(topicName, objectWriter.writeValueAsString(payload).getBytes());
        } catch (JsonProcessingException e) {
            log.info("error converting object into byte array {}", payload.toString().substring(0, 50));
        }
        log.info("sent payload to topic='{}'", topicName);
    }
}

But when I send the message using kafkaTemplate to any topic, StreamListener doesn't receive the message.

spring.cloud.stream.bindings.input.group=test
spring.cloud.stream.bindings.input.destination=eventUpdateFromEventModel

my pom.xml:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-stream-kafka</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-test-support</artifactId>
    <scope>test</scope>
</dependency>

    <!-- Spring boot version -->
    <spring.boot.version>1.5.7.RELEASE</spring.boot.version>
    <spring-cloud.version>Edgware.SR3</spring-cloud.version>

<dependencyManagement>
      <dependencies>

        <dependency>
            <!-- Import dependency management from Spring Boot -->
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-dependencies</artifactId>
            <version>${spring.boot.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-dependencies</artifactId>
            <version>${spring-cloud.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>

    </dependencies>
</dependencyManagement>

Answer 1

working

Object payload = messageCollector.forChannel(pipe.output())
        .poll()
        .getPayload();

...

not working

KafkaTemplate

This is because you are using the TestBinder in your test, not the real Kafka broker and kafka binder.

The message collector is simply fetching it from the channel. If you want to test with a real Kafka broker, see the test-embedded-kafka sample app .

EDIT

I just tested the Ditmars (boot 1.5.x) version of the sample and it works fine...