[英]Kafka Avro Deserializer is not able to deserialize the Kafka message to Specific Avro Record
[英]Decrypt Kafka Avro Message
我正在尝试解析kafka消息,它采用某种加密的AVRO格式。 我有以下AvroSchema.avsc avro模式文件:
{
"type": "record",
"namespace": "kafka.events",
"name": "AvroSchema",
"fields": [
{ "name": "product_id", "type": "string" },
{ "name": "available_to_promise_quantity", "type": "double" },
{ "name": "online_available_to_promise_quantity", "type": "double" },
{ "name": "stores_available_to_promise_quantity", "type": "double" },
{ "name": "is_infinite_inventory", "type": "boolean", "default" : false },
{ "name": "event_timestamp", "type": "long" },
{ "name": "previous_event", "type": "AvroSchema" }
]
}
现在,我编写了以下代码以JSON格式获取数据:
for (final KafkaStream<byte[], byte[]> stream : streams){
ConsumerIterator<byte[], byte[]> consumerIterator = stream.iterator();
byte[] consumedEncryptedMessage;
MessageAndMetadata<byte[], byte[]> consumedEntry;
while(consumerIterator.hasNext()){
consumedEntry = consumerIterator.next();
if(null != consumedEntry){
consumedEncryptedMessage = consumedEntry.message();
try {
Schema schema = null;
schema = new Schema.Parser().parse(new File("src/AvroSchema.avsc"));
DatumReader<GenericRecord> reader = new SpecificDatumReader<GenericRecord>(schema);
Decoder decoder = DecoderFactory.get().binaryDecoder(consumedEncryptedMessage , null);
GenericRecord decryptedmsg = null;
decryptedmsg = reader.read(null, decoder);
System.out.println(decryptedmsg);
}
catch(Exception e) {
e.printStackTrace();
System.out.println(e);
}
请帮助我如何解密邮件。
加密的字节消息具有以下类型: 080-21-0001 : Aw @@ A ǐ U : Aw @@ A
我按照建议进行了更改,现在有以下代码:
while(consumerIterator.hasNext()){
consumedEntry = consumerIterator.next();
if(null != consumedEntry){
consumedEncryptedMessage = consumedEntry.message();
try {
Schema schema = new Schema.Parser().parse(new File("src/AVROSchema.avsc"));
File myfile = new File("/Users/z001ldc/Desktop/myfile.txt");
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema);
FileUtils.writeByteArrayToFile(myfile, consumedEncryptedMessage);
@SuppressWarnings("resource")
DataFileReader<GenericRecord> dataFileReader = new DataFileReader<GenericRecord>(myfile, reader);
while (dataFileReader.hasNext()) {
decryptedMessage = dataFileReader.next(decryptedMessage);
System.out.println(decryptedMessage.get("product_id").toString());
}
}
catch(Exception e) {
e.printStackTrace();
System.out.println(e);
}
但是,我仍然收到错误提示为“不是数据文件”。
反序列化不需要解密
首先,您获得了架构行
schema = new Schema.Parser().parse(new File("src/AvroSchema.avsc"));
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema);
然后
DataFileReader<GenericRecord> dataFileReader = new DataFileReader<GenericRecord>(consumedEncryptedMessage, reader);
GenericRecord user = null;
while (dataFileReader.hasNext()) {
// Reuse user object by passing it to next(). This saves us from
// allocating and garbage collecting many objects for files with
// many items.
user = dataFileReader.next(user);
System.out.println(user);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.