处理模式注册表deserialiser kafka时出现异常

Question

I'm following the confluent link to post kafka messages on changes to mysql table. When I try to consume this message from a springboot application, I'm getting the below exception. How can I fix this so that I can read the message. Sometimes I am able to read the message but I get a serialised version if like consumed message is key: null value : 2foo 䶰Z 䶰Z

Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to com.spring.kafkaexample.springbootkafkaconsumer.model.Foobar
    at com.spring.kafkaexample.springbootkafkaconsumer.listener.KafkaConsumer.consume(KafkaConsumer.java:22) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:181) ~[spring-messaging-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:114) ~[spring-messaging-5.0.8.RELEASE.jar:5.0.8.RELEASE]

Below is my code

@KafkaListener(topics = "mysql-foobar", groupId = "group_id")
public void consume(ConsumerRecord<String, Foobar> message) {

    System.out.println("consumed message is key: "+message.key() + " value :  "+message.value());
}

Deserializer

@Bean
public Map<String, Object> consumerConfigs() {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaAvroDeserializer.class);
    props.put(ConsumerConfig.GROUP_ID_CONFIG, "avro");

    return props;
}

Answer 1

If you are getting String cannot be cast to com.spring.kafkaexample.springbootkafkaconsumer.model.Foobar , then that means you have a producer sending a string into your topic, which is not a valid to be made a Foobar instance.

If you are getting the 2foo 䶰Z 䶰Z stuff, then that would mean you're printing out the UTF-8 raw bytes of some binary data, such as Avro, and you would be better off extracting the fields of the object rather than relying solely on the implicit toString method.