Serialize in C# (protobuf-net) , Deserialize in C++ (protobuf) : More than 5 fields in class

Question

I'm having trouble deserializing an object in C++ that I had serialized in C# and then sent over the network with ZMQ. I'm fairly certain the ZMQ part is working correctly because the C++ server application (Linux) successfully receives the serialized messages from C# (Windows) and sends them back to Windows where it can successfully deserialize the message, so I don't think I'm experiencing any sort of truncated or dropped packets in that regard.

However, when I receive the message on the Linux server, the C++ deserialize method does not correctly deserialize, it throws some a bunch of binary data into the 6th field (I can see this in MyObject.DebugString()), but no data in any other fields. The strange part here, however, is that a class I had with 5 fields works perfectly fine. C++ deserializes it correctly and all of the data is working properly. Below are a few tidbits of my code. Any help would be greatly appreciated.

C#:
    MemoryStream stream = new MemoryStream();
    ProtoBuf.Serializer.Serialize<TestType>(stream, (TestType)data);
    _publisher.Send(stream.ToArray());

C++:
    message_t data;
    int64_t recv_more;
    size_t recv_more_sz = sizeof(recv_more);
    TestType t;
    bool isProcessing = true;
    while(isProcessing)
    {
      pSubscriber->recv(&data, 0);
      t.ParseFromArray((void*)(data.data()),sizeof(t));
      cout<<"Debug: "<<t.DebugString()<<endl;  

      pSubscriber->getsockopt(ZMQ_RCVMORE, &recv_more, &recv_more_sz);
      isProcessing = recv_more;
    }

The output looks like this:

Debug: f: "4\000\000\000\000\000\"

I'm having trouble copy and pasting, but the output continues like that for probably 3 or 4 lines worth of that.

This is my TestType class (proto file):

package Base_Types;

enum Enumr {
  Dog = 0;
  Cat = 1;
  Fish = 2;
}

message TestType {
  required double a = 1;
  required Enumr b = 2;
  required string c = 3;
  required string d = 4;
  required double e = 5;
  required bytes f = 6;
  required string g = 7;
  required string h = 8;
  required string i = 9;
  required string j = 10;
}

Field "f" is listed as bytes because when it was a string before it was giving me a warning about UTF-8 encoding, however, when this class worked with only 5 fields (the enum was one of them), it did not give me that error. It's almost like instead of deserializing, it's throwing the binary for the entire class into field "f" (field 6).

Solution: There ended up being an issue where the memory wasn't being copied before it sent to a thread socket. When the publisher sent back out, it was packaging the data and changing what the router received. There needs to be a memcpy() on the C++ side in order to send out the data to be used internally. Thanks for all of the help.

Answer 1

I've parsed it through the reader in v2, and it seems to make perfect sense:

1=5
2=0
3=
4=yo
5=6
6=2 bytes, 68-69
7=how
8=are
9=you
10=sir

Note that I've done that purely from the hex data (not using the .proto), but it should be close to your original data. But most notably, it seems intact.

So: first thing to do; check that the binary you get at the C++ side is exactly the same as the binary you sent; this is doubly important if you are doing any translations along the way (binary => string, for example - which should be done via base-64).

second thing; if that doesn't work, it is possible that there is a problem in the C++ implementation. It seems unlikely since that is one of google's pets, but nothing is impossible. If the binary comes across intact, but it still behaves oddly, I can try speaking to the C++ folks, to see if one of us has gone cuckoo.