[英]Issue in deserialize protostuff object in Kafka Consumer
Getting following exception while deserializing byte[] into protostuff object in Kafka Consumer 在Kafka Consumer中将byte []反序列化为原型对象时遇到以下异常
java.lang.NegativeArraySizeException
at com.dyuproject.protostuff.GraphIOUtil.mergeDelimitedFrom(GraphIOUtil.java:209)
at com.gme.protocols.protostuff.GmeTrade.readExternal(GmeTrade.java:2772)
at java.io.ObjectInputStream.readExternalData(Unknown Source)
at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
Converted protostuff object to byte[] using following code. 使用以下代码将protostuff对象转换为byte []。
public static byte[] toBytes(Object o)
{
try
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(o);
oos.close();
byte[] b = baos.toByteArray();
return b;
}
catch (IOException e)
{
return new byte[0];
}
}
Sent that byte[] using Kafka producer with topic 'XX', where byte[] length is just 240. Received that record using Kafka consumer. 使用主题为“ XX”的Kafka生产者发送了byte [],其中byte []长度仅为240。使用Kafka消费者接收了该记录。 record.value().length (byte[]) length is same 240 what I sent from producer side. record.value()。length(byte [])的长度与我从生产者端发送的长度相同240。
Deserialized that byte[] to object using following code. 使用以下代码将该byte []反序列化为对象。
public static Object fromBytes(byte[] bytes)
{
try
{
return new ObjectInputStream(new ByteArrayInputStream(bytes)).readObject();
}
catch (Exception e)
{
e.printStackTrace();
return null;
}
}
Getting above mentioned exception. 获得上述异常。 What I am doing wrong ? 我做错了什么? Using kafka_2.11-0.9.0.0 for your reference. 使用kafka_2.11-0.9.0.0供参考。 Is there any other things needed? 还有其他需要的东西吗?
I found the solution. 我找到了解决方案。 Issue is there in toBytes and fromBytes. 在toBytes和fromBytes中存在问题。 Need to convert it into byte[] using ProtostuffIOUtil.toByteArray method. 需要使用ProtostuffIOUtil.toByteArray方法将其转换为byte []。
Serialization 序列化
public static byte[] toBytes(Foo o)
{
LinkedBuffer BUFFER = LinkedBuffer.allocate(1024*1024);
Schema<Foo> SCHEMA = Foo.getSchema();
return ProtostuffIOUtil.toByteArray(o, SCHEMA, BUFFER);
}
Again need to convert byte[] to object using ProtostuffIOUtil.mergeFrom method. 再次需要使用ProtostuffIOUtil.mergeFrom方法将byte []转换为对象。
Deserialization 反序列化
public static Foo fromBytes(byte[] bytes)
{
Foo tmp = Foo.getSchema().newMessage();
ProtostuffIOUtil.mergeFrom(bytes, tmp, Foo.getSchema());
return tmp;
}
Note : Serialization/Deserialization with ObjectOutputStream/ObjectInputStream will not work for protostuff objects. 注意:使用ObjectOutputStream / ObjectInputStream进行序列化/反序列化不适用于原型对象。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.