使用协议缓冲区将二进制文件从Java Server发送到C＃Unity3d客户端

Question

I have asked this question https://stackoverflow.com/questions/32735189/sending-files-from-java-server-to-unity3d-c-sharp-client but I saw that it isn't an optimal solution to send files between Java and C# via built-in operations, because I also need also other messages, not only the file content.

Therefore, I tried using Protobuf, because it is fast and can serialize/deserialize objects platform independent. My .proto file is the following:

message File{
  optional int32 fileSize = 1;
  optional string fileName = 2;
  optional bytes fileContent = 3;
}

So, I set the values for each variable in the generated .java file:

file.setFileSize(fileSize);
file.setFileName(fileName);
file.setFileContent(ByteString.copyFrom(fileContent, 0, fileContent.length);

I saw many tutorials about how to write the objects to a file and read from it. However, I can't find any example about how to send a file from server socket to client socket.

My intention is to serialize the object (file size, file name and file content) on the java server and to send these information to the C# client. So, the file can be deserialized and stored on the client side.

In my example code above, the server read the bytes of the file (image file) and write it to the output stream, so that the client can read and write the bytes to disk through input stream. I want to achieve the same thing with serialization of my generated .proto file.

Can anyone provide me an example or give me a hint how to do that?

Answer 1

As described in the documentation , protobuf does not take care of where a message start and stops, so when using a stream socket like TCP you'll have to do that yourself.

From the doc:

[...] If you want to write multiple messages to a single file or stream, it is up to you to keep track of where one message ends and the next begins. The Protocol Buffer wire format is not self-delimiting, so protocol buffer parsers cannot determine where a message ends on their own. The easiest way to solve this problem is to write the size of each message before you write the message itself. When you read the messages back in, you read the size, then read the bytes into a separate buffer, then parse from that buffer. [...]

Length-prefixing is a good candidate. Depending on what language you're writing, there are libraries that does length-prefixing for eg TCP that you can use, or you can define it yourself.

An example representation of the buffer on the wire might beof the format might be (beginning of buffer to the left):

[buf_length|serialized_buffer2]

So you code to pack the the buffer before sending might look something like (this is in javascript with node.js):

function pack(message) {
  var packet = new Buffer(message.length + 2);

  packet.writeIntBE(message.length, 0, 2);
  message.copy(packet, 2);

  return packet;
}

To read you would have to do the opposite:

client.on('data', function (data) {
  dataBuffer = Buffer.concat([dataBuffer, data]);

  var dataLen = dataBuffer.readIntBE(0, 2);

  while(dataBuffer.length >= dataLen) {
    // Message length excluding length prefix of 2 bytes
    var msgLen = dataBuffer.readIntBE(0, 2);

    var thisMsg = new Buffer(dataBuffer.slice(2, msgLen + 2));

    //do something with the msg here

    // Remove processed message from buffer
    dataBuffer = dataBuffer.slice(msgLen + 2);
  }
});

You should also be aware of that when sending multiple protobufs on a TCP socket, they are likely to be buffered for network optimizations (concatenated) and sent together. Meaning some sort of delimiter is needed anyway.