从kafka流和Avro反序列化同一类的ClassCastException

Question

I have generated my Avro Java classes from Avro schema with avro-maven-plugin. I serialize my avro class to a byte array and I write it into a kafka topic.

Then I have a kafka stream that tries to manipulate avro data to do something. During the deserialization process I get a ClassCastExcetion from the same class. I read that this problem is generated due to a different ClassLoader that Avro uses on fallback (a new instance of ClassLoader).

There is a method to force Avro to use the caller's ClassLoader or something similar?

KafkaStream properties

this.props = new Properties();
        this.props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        this.props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        this.props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
        this.props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.ByteArray().getClass());

I'm using a String key and the serialized avro's byte array, then I need to manually deserialize the avro's payload. I use avro's decoder to deserialize like this:

AvroPayload stp = AvroPayload.fromByteBuffer(ByteBuffer.wrap(bytes));

or even like this:

AvroPayload stp = AvroPayload.getDecoder().decode(ByteBuffer.wrap(bytes));

With the first version, debugging I can see that if I remain in the avro's generated class context the byte array is correctly deserialized into the AvroPayload class. Returning that new instance perhaps throws a ClassCastException

Answer 1

The only solution I found is to put as suggested the avro classes into an external jar and then import it.

It's not a good solution because it requires a lot of configuration to keep coupled your avro schemas and your generated classes, but it's the only I found.

I configured a maven project that generates the avro's classes jar into a directory, without artifact version in its name, so I can always import the latest version without having to change the pom.

If someone will discover another solution, please post it