[英]Avro Map Serialization/Deserialization issue
Currently I am using Avro 1.8.0 to serialize / deserialize objects but facing issue especially for java.util.Map object. 目前,我正在使用Avro 1.8.0来序列化/反序列化对象,但是尤其是对于java.util.Map对象,却遇到了问题。 Not facing issue with other type of objects.
不会遇到其他类型的对象的问题。
Sample code here - 示例代码在这里-
class AvroUtils {
public byte[] serialize(Object payload) {
final ByteArrayOutputStream out = new ByteArrayOutputStream();
Schema schema = new ReflectDatumWriter().getData().induce(payload); //---> getting proper map schema as {"type":"map","values":"string"}
JsonEncoder jsonEncoder = EncoderFactory.get().jsonEncoder(schema, out);
final GenericDatumWriter<Object> writer = new GenericDatumWriter(schema);
writer.write(payload, jsonEncoder);
jsonEncoder.flush();
return out.toByteArray();
}
public <R> R deserialize(Object o, Class<R> aClass) {
Schema schema = new ReflectDatumWriter().getData().induce(o); //------> getting error - unable to get schema
final ByteArrayInputStream bin = new ByteArrayInputStream((byte[]) o);
JsonDecoder jsonDecoder = DecoderFactory.get().jsonDecoder(schema, bin);
final GenericDatumReader<R> reader = new GenericDatumReader<>(schema);
return reader.read(null, jsonDecoder);
}
public static void main(String[] args) {
Map<String, Object> map = new HashMap<String, Object>();
map.put("Key1", "Value1");
map.put("Key2", "Value2");
// Serialization
byte[] result = this.serialize(map);
System.out.println("Serialized Data : " + new String(mapDes, "UTF-8"));
// Deserialization
Map<String, Object> mapDes = (Map<String, Object>) this.deserialize(result, Map.class);
System.out.println("Deserialized Data : " + mapDes);
}
}
In deserialize method I am trying to get the schema based on input data but avro is throwing error - 在反序列化方法中,我试图基于输入数据获取架构,但是avro抛出错误-
`Exception in thread "main" java.lang.ClassCastException: [B cannot be cast to java.util.Collection
at org.apache.avro.reflect.ReflectData.getArrayAsCollection(ReflectData.java:196)
at org.apache.avro.generic.GenericData.induce(GenericData.java:612)`
Note: At the end both the methods will be placed in different libraries (avro-serializer / avro-deserializer). 注意:最后,两种方法都将放置在不同的库中(avro-serializer / avro-deserializer)。
Please suggest the best way to get schema in deserialization method. 请提出反序列化方法中获取架构的最佳方法。
Thanks. 谢谢。
You get java.lang.ClassCastException: [B cannot be cast to java.util.Collection
cause you try to call induce()
method with object which is byte array but not a Map. 您会收到
java.lang.ClassCastException: [B cannot be cast to java.util.Collection
因为您尝试使用字节数组而不是Map的对象调用induce()
方法。
If you want to serialize a map in one place and deserialize it in the other, you can use a better way: 如果要在一个位置序列化地图,在另一个位置反序列化地图,则可以使用更好的方法:
Schema schema = Schema.createMap(Schema.create(Schema.Type.STRING));
If you do so, you will not need any additional parameters in desirialize method. 如果这样做,则在desirialize方法中不需要任何其他参数。
Also, GenericDatumWriter
can be used only with generic records, so you need a ReflectDatumWriter
. 另外,
GenericDatumWriter
只能与通用记录一起使用,因此您需要ReflectDatumWriter
。
Here is an example of your code with changes: 这是一个带有更改的代码示例:
public class AvroUtils {
public static byte[] serialize(Object payload) throws IOException {
final ByteArrayOutputStream out = new ByteArrayOutputStream();
Schema schema = Schema.createMap(Schema.create(Schema.Type.STRING)); //---> getting proper map schema as {"type":"map","values":"string"}
JsonEncoder jsonEncoder = EncoderFactory.get().jsonEncoder(schema, out);
final DatumWriter<Object> writer = new ReflectDatumWriter<>(schema);
writer.write(payload, jsonEncoder);
jsonEncoder.flush();
return out.toByteArray();
}
public static <R> R deserialize(Object o) throws IOException {
Schema schema = Schema.createMap(Schema.create(Schema.Type.STRING));
JsonDecoder jsonDecoder = DecoderFactory.get().jsonDecoder(schema, new ByteArrayInputStream((byte[]) o));
final DatumReader<R> reader = new ReflectDatumReader<>(schema);
return reader.read(null, jsonDecoder);
}
public static void main(String[] args) throws IOException {
Map<String, Object> map = new HashMap<>();
map.put("Key1", "Value1");
map.put("Key2", "Value2");
// Serialization
byte[] result = serialize(map);
// Deserialization
Map<String, Object> mapDes = deserialize(result);
System.out.println("Deserialized Data : " + mapDes);
}
}
As a result, you'll get something like this: 结果,您将获得以下内容:
Deserialized Data : {Key2=Value2, Key1=Value1}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.