SerializationException: Error deserializing Avro message (StringIndexOutOfBoundsException)

Question

I've run into this error when KafkaStream tries to deserialise the Arvo message:

[filtering-app-6adef284-11eb-48f8-8ca0-cde7da5224ab-StreamThread-1] ERROR org.apache.kafka.streams.KafkaStreams - stream-client [filtering-app-6adef284-11eb-48f8-8ca0-cde7da5224ab] All stream threads have died. The instance will be in error state and should be closed.
[filtering-app-6adef284-11eb-48f8-8ca0-cde7da5224ab-StreamThread-1] INFO org.apache.kafka.streams.processor.internals.StreamThread - stream-thread [filtering-app-6adef284-11eb-48f8-8ca0-cde7da5224ab-StreamThread-1] Shutdown complete
Exception in thread "filtering-app-6adef284-11eb-48f8-8ca0-cde7da5224ab-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: Deserialization exception handler is set to fail upon a deserialization error. If you would rather have the streaming pipeline continue after a deserialization error, please set the default.deserialization.exception.handler appropriately.
    at org.apache.kafka.streams.processor.internals.RecordDeserializer.deserialize(RecordDeserializer.java:80)
    at org.apache.kafka.streams.processor.internals.RecordQueue.maybeUpdateTimestamp(RecordQueue.java:160)

The cause exception was:

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 1
Caused by: java.lang.RuntimeException: java.lang.StringIndexOutOfBoundsException: begin 1, end 0, length 1
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1529)

and

Caused by: java.lang.StringIndexOutOfBoundsException: begin 1, end 0, length 1
    at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
    at java.base/java.lang.String.substring(String.java:1874)

The avro configuration is straigthforward:

{
  "namespace": "io.confluent.developer.avro",
  "type": "record",
  "name": "Publication",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "title", "type": "string"}
  ]
}

which is from this tutorial: https://kafka-tutorials.confluent.io/filter-a-stream-of-events/kstreams.html . The producer serialises the input string "{"name": "George R. R. Martin", "title": "A Dream of Spring"}" with no problem, but then the KafkaStream which basically tries to filter the event failed to deserialise the object to perform the Java filtering logic...

Has anyone encountered this problem before? Appreciate any suggestions!

Answer 1

Found the issue: a proxy gets in the way.

The root cause was that the app can't connect to schema-registry. Just note it here in case someone runs into the same problem later.