[英]Kafka Streams Schema Issue when Branching to Multiple Topics
I have an application which branches to one of 20 topics based on some rules.我有一个应用程序,它根据一些规则分支到 20 个主题之一。 It has starded failing for below error.它因以下错误而开始失败。
stream-thread [pbel-topic-splitter-bc7189fd-e35c-454d-b5e7-97d5e514dc36-StreamThread-1] Failed to process stream task 0_14 due to the following error:
org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_14, processor=KSTREAM-SOURCE-0000000000, topic=com.bns.prm. raw.DETAIL, partition=14, offset=334
at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:367)
at org.apache.kafka.streams.processor.internals.AssignedStreamsTasks.process(AssignedStreamsTasks.java:104)
at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:413)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:862)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:777)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:747)
Caused by: org.apache.kafka.common.errors.SerializationException: Error registering Avro schema: {"type":"record","name":"DETAIL","namespace":"com. bns.eventlayer.prm.avro","fields":[{"name":"isb_IDENTITY_NUMBER","type":"long"},{"name":"ND_TRAN_ID","type":["null","long"],"default":null},{"name" :"D nnect.data.Timestamp","logicalType":"timestamp-millis"}}],"connect.name":"com.bns.eventlayer.prm.avro.DETAIL"}
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Schema being registered is incompatible with an earlier schema; error code: 409
at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:209)
I compared the 2 schemas and found this subtle difference, not sure if this matters...我比较了这两种模式,发现了这种微妙的差异,不确定这是否重要......
{
"name":"SD_ACH_BATCH_OFFSET_TYP",
"type":[
"null",
"string"
],
"default":null
}
{
"name":"SD_SECURITY_LEVEL",
"type":[
"null",
{
"type":"string",
"avro.java.string":"String"
}
],
"default":null
},
its was my mistake, i had to clear an old schema on error topic .这是我的错误,我必须清除错误主题的旧模式。 Wrote a program to seek to the offset which always crashed (DLQ), read data of the DLQ and figured out its going to the error topic...编写了一个程序来寻找总是崩溃的偏移量(DLQ),读取 DLQ 的数据并找出它的错误主题......
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.