Spring Cloud Kinesis Binder How to Handle ERROR for Producer - As per Documentation it is Not working

Question

I have followed the below documentation and I have a producer and Consumer working perfectly fine with Kinesis Stream. I would like to understand how to handle the ERROR in Producer (Source) in case of any exception happens.

I have tried below approaches as per Spring Stream Error Handling Documentations:

@StreamListener("errorChannel")
@ServiceActivator(inputChannel = "errorChannel")

Both are not working. I am throwing a RuntimeException explicitly in the producer method and expecting that will be in the "errorChannel" but unable to receive the same.

Please help me to figure out this or share it with me approach if someone successfully did this.

Answer 1

Note: I have come across this exact problem few months ago. We were using Kafka in place of Kinesis. Since, you have tagged spring-cloud-stream-binder-kafka, I am giving the inputs on the same ground.

You can write a custom call back for your producer and this call back can tell you if the message has failed or successfully published. On failure, log the meta data for the message.

Below is a small code snippet than can explain better about the call back I was talking about.

Please note that this is for Kafka. Change the code accordingly for Kinesis.

public class ProduceToKafka {

  private ProducerRecord<String, String> message = null;

 // TracerBulletProducer class has producer properties
  private KafkaProducer<String, String> myProducer = TracerBulletProducer
      .createProducer();

  public void publishMessage(String string) {

    ProducerRecord<String, String> message = new ProducerRecord<>(
        "topicName", string);

    myProducer.send(message, new MyCallback(message.key(), message.value()));
  }

  class MyCallback implements Callback {

    private final String key;
    private final String value;

    public MyCallback(String key, String value) {
      this.key = key;
      this.value = value;
    }


    @Override
    public void onCompletion(RecordMetadata metadata, Exception exception) {
      if (exception == null) {
        log.info("--------> All good !!");
      } else {
        log.info("--------> not so good  !!");
        log.info(metadata.toString());
        log.info("" + metadata.serializedValueSize());
        log.info(exception.getMessage());

      }
    }
  }

}

I wrote a medium article of handling the producer failures at different levels of Spring abstractions. This can help you. Check it out : https://medium.com/@akhil.ghatiki/kafka-producer-failure-handling-at-multiple-levels-of-spring-abstractions-e530edb02a6c