在jackson生成的json中包装协议缓冲区时java中的错误？

Question

I'm sending and receiving HTTP post requests by doing the following:

FooJson fooJson = new FooJson();
fooJson.setName("Bob");

FooProto.Builder fooProto = FooProto.newBuilder(); // google protobuf
fooProto.setColor("Blue");
fooProto.setLength(30);

BarProto.Builder barProto = BarProto.newBuilder();
barProto.setWeight(65);
fooProto.setBarProto(barProto);

barJson.setFooProto(new String(fooProto.build().toByteArray()));
List<BarJson> barJsonList = new ArrayList<BarJson>();
barJsonList.add(barJson);
fooJson.setBarJsonList(barJsonList);
String data = writeJson(fooJson); // wrapper for jackson JsonGenerator
RequestEntity re = new ByteArrayRequestEntity(data.getBytes());

PostMethod method = new PostMethod("http://foo.net:123/path");
method.setRequestEntity(re);
httpClient.executeMethod(method);

On the receiving end, I parse with the following:

FooJson fooJson = readJson(request.getInputStream(), FooJson.class);
for (BarJson barJson : fooJson.getBarJsonList()) {
    FooProto fooProto = FooProto.parseFrom(barJson.getFooProto().getBytes());
}

The result during parsing of the protocol buffer on the receiving end is:

com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either than the input has been truncated or that an embedded message misreported its own length.

Am I doing something wrong with the way I'm converting the protocol buffer to a string? How might I be able to fix this?

Answer 1

I doubt that you can reliably tunnel protobuf (whose payload is pure binary, not text) through JSON without doing some type of encoding like base64 or hex encoding. Convert your protobuf payload from bytes into base64 text. Then on the receiving end convert it from base64 text back to binary.

You are getting the protobuf exception because the byte array on the receiving end does not match the protobuf payload you sent. The data got munged when you converted to a string and back without using some type of encoding.

I have had good luck with javax.xml.bind.DatatypeConverter. It's part of Java 1.6. Use printBase64Binary on the sending side and parseBase64Binary on the receiving side.

[update]

Alternatively, if base64 is just too ugly, you can serialize your protobuf objects into several different string formats (JSON, XML) with protobuf-java-format . This might look a little weird because the barJson.setFooProto would contain a string which is itself a JSON payload. There would be a lot of escaped quote characters - but it should work.

Answer 2

Your mistake is here:

barJson.setFooProto(new String(fooProto.build().toByteArray()));

Do NOT try to create a String from arbitrary binary data -- this will quite likely corrupt the data, depending on encodings in use; and specifically with JSON, UTF-8 encoding will pretty much guarantee corruption (some byte sequence are not allowed in UTF-8, so you either get an exception or data gets mutated).

Instead, define field as byte[] , and let Jackson use Base64 encoding. This will be correct, and is done relatively efficiently.

You might also want to consider just using JSON for the whole thing, as the first answer suggested. No need to use specific Protobuf flavor either; you can always construct PB objects separately if you really need them.