[英]Spring boot kafka with schema registry - payload is not matching at consumer end
I have a producer with this configuration我有一个具有这种配置的生产者
kafka:
bootstrap-servers: localhost:9092
cloud:
stream:
binder:
consumer-properties:
key.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
value.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
schema.registry.url: http://localhost:8081
properties:
specific.avro.reader: true
schemaRegistryClient:
endpoint: http://localhost:8081
bindings:
event-in-0:
destination: event-details
contentType: application/*+avro
group: group1
function:
definition: event
This is my schema这是我的架构
{
"type": "record",
"name": "Event",
"namespace": "com.example.schema.avro",
"fields": [
{
"name": "eventId",
"type": "int"
}
]
}
and code to publish the message和发布消息的代码
public void sendEvent(final EventDto eventDto) {
final Event apply = event().apply(eventDto);
final Message<Event> build = MessageBuilder.withPayload(apply)
.setHeader("partitionKey", eventDto.eventId())
.setHeader("customHeader", "test").build();
final boolean send = streamBridge.send("event-out-0", build);
log.info(String.valueOf(build));
}
This produces the correct event as seen in the logs这会产生正确的事件,如日志中所示
2022-08-15 09:12:23.988 INFO 25112 --- [nio-9090-exec-5] c.e.eventproducer.service.EventService : GenericMessage [payload={"eventId": 200}, headers={customHeader=test, id=70c3e52c-f419-cf0c-8fae-0ba07d8876da, partitionKey=200, timestamp=1660547543987}]
Now at the consumer end, I expect the same eventId:200 but I always get 0 no matter what the event id is.现在在消费者端,我期望相同的 eventId:200 但无论事件 id 是什么,我总是得到 0。 This is my consumer configuration这是我的消费者配置
kafka:
bootstrap-servers: localhost:9092
cloud:
stream:
binder:
consumer-properties:
key.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
value.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
schema.registry.url: http://localhost:8081
properties:
specific.avro.reader: true
schemaRegistryClient:
endpoint: http://localhost:8081
bindings:
event-in-0:
destination: event-details
contentType: application/*+avro
group: group1
function:
definition: event
This is to subscribe the message这是订阅消息
@Bean
public Consumer<Message<Event>> event() {
return e -> {
log.info(e.toString());
eventRepository.save(new com.example.eventconsumer.doamin.Event(e.getPayload().getEventId()));
};
}
and the log shows日志显示
2022-08-15 09:12:23.992 INFO [,b19cd949219432acf232403bdcea45c2,1464ff27cfed8011] 24596 --- [container-0-C-1] c.e.eventconsumer.service.EventService : GenericMessage [payload={"eventId": 0}, headers={customHeader=test, deliveryAttempt=1, kafka_timestampType=CREATE_TIME, scst_partition=0, kafka_receivedTopic=event-details, kafka_offset=97, partitionKey=200, scst_nativeHeadersPresent=true, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@41d9d95, source-type=kafka, id=0762521f-069b-a91d-acc5-69e6bb2eb4eb, kafka_receivedPartitionId=0, contentType=application/vnd.event.v1+avro, kafka_receivedTimestamp=1660547543987, kafka_groupId=group1, timestamp=1660547543992}]
Interestingly if I pass the partition key as event id then I fetch it correctly but not the payload itself.有趣的是,如果我将分区键作为事件 id 传递,那么我会正确获取它,但不会获取有效负载本身。
I believe, the issue is with your DTO, where the value of the evenId is not setting/getting correctly and it is always returning the default value of int, which is 0 .我相信,问题出在您的 DTO 上,其中 evenId 的值未正确设置/获取,并且它始终返回 int 的默认值,即0 。
This is how it looks the auto-generated class out of your.avsc.这是从 your.avsc 中自动生成的 class 的外观。 So, we need to set/get eventId in your event class appropriately.因此,我们需要在您的事件 class 中适当地设置/获取 eventId。
/** Gets the value of the 'eventId' field */
public java.lang.Integer getEventId() {
return eventId;
}
/** Sets the value of the 'eventId' field */
public com.example.schema.avro.Event.Builder setEventId(int value) {
validate(fields()[0], value);
this.eventId = value;
fieldSetFlags()[0] = true;
return this;
}
public Event build() {
try {
Event record = new Event();
record.eventId = fieldSetFlags()[0] ? this.eventId : (java.lang.Integer) defaultValue(fields()[0]);
return record;
} catch (Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.