[英]How can you know the length of the byte-array to deserialize using protobuf-net?
I am trying to send a C# object from one process to another using a Memory-Mapped File, and I am trying to use either BinaryFormatter or protobuf-net. 我正在尝试使用内存映射文件将C#对象从一个进程发送到另一个进程,并且试图使用BinaryFormatter或protobuf-net。 Neither is working - apparently because I am using by necessity a fixed-length byte-array and protobuf-net needs an array that is exactly the correct length?
都不起作用-显然是因为我必须使用固定长度的字节数组,而protobuf-net需要一个长度正确的数组?
Using protobuf-net, I get this exception on Deserialize: "ProtoException: 'Unconsumed data left in the buffer; this suggests corrupt input' at the line: "message1 = Serializer.Deserialize(memoryStream); 使用protobuf-net,我在Deserialize上收到此异常:“ ProtoException:'未使用的数据留在缓冲区中;这表明输入内容已损坏'。”:“ message1 = Serializer.Deserialize(memoryStream);”。
Here is my code. 这是我的代码。 At this point I'm just trying a simple example in order to get it to work at a basic level: This is the object that I want to send between programs:
在这一点上,我只是尝试一个简单的示例,以使其在基本级别上起作用:这是我想在程序之间发送的对象:
[ProtoContract]
public class IpcMessage
{
public IpcMessage() { }
[ProtoMember(1)]
public string title { get; set; }
[ProtoMember( 2 )]
public string content { get; set; }
}
Here is the (simplified - I removed the synchronization) code that sends the IpcMessage object: 这是发送IpcMessage对象的(简化后,我删除了同步)代码:
static void SampleSend()
{
// Create the memory-mapped file which allows 'Reading' and 'Writing'
using (MemoryMappedFile mmf = MemoryMappedFile.CreateOrOpen( "MyMmfName", 1024, MemoryMappedFileAccess.ReadWrite ))
{
// Create a view-stream for this process, which allows us to write data from offset 0 to 1024 (whole memory)
using (MemoryMappedViewStream mmvStream = mmf.CreateViewStream( 0, 1024))
{
IpcMessage message1 = new IpcMessage();
message1.title = "test";
message1.content = "hello world";
Serializer.Serialize( mmvStream, message1 );
}
}
}
and here is the code in the receiving program (simplified a bit): 这是接收程序中的代码(简化了一点):
// Create the memory mapped file..
using (MemoryMappedFile mmf = MemoryMappedFile.CreateOrOpen( "MyMmfName", 1024, MemoryMappedFileAccess.ReadWrite ))
{
using (MemoryMappedViewAccessor mmvStream = mmf.CreateViewAccessor( 0, 1024, MemoryMappedFileAccess.Read ))
{
byte[] buffer = new byte[1024];
IpcMessage message1;
int numberBytesRead = mmvStream.ReadArray<byte>( 0, buffer, 0, 1024 );
var memoryStream = new MemoryStream(buffer);
// It is at this next line that I get ProtoException: 'Unconsumed data left in the buffer; this suggests corrupt input'
message1 = Serializer.Deserialize<IpcMessage>( memoryStream );
}
}
When I was trying to use BinaryFormatter it complained also. 当我尝试使用BinaryFormatter时,它也抱怨。 Clearly I am doing something very basically wrong.
显然,我所做的事情基本上是错误的。
Looking thru other questions - I see that most implementations seem to have a byte-array that is allocated to just the right length. 通过其他问题来看-我看到大多数实现似乎都有一个分配给正确长度的字节数组。 Here, I don't know the length ahead of time -- it is just a fixed-length 1024-byte array (I chose this size arbitrarily, just for now).
在这里,我不知道提前的长度-它只是一个固定长度的1024字节数组(我暂时选择了这个大小)。 Or perhaps it's something obvious that I'm missing?
还是很明显我想念的东西?
This is my first time to use a Memory-Mapped File, or protobuf-net. 这是我第一次使用“内存映射文件”或protobuf-net。 Any help or advice is appreciated - thank you in advance.
感谢您提供任何帮助或建议-预先感谢您。
Note: I'm using Visual Studio 2017 Enterprise 15.9.6, and this code is targeting .NET Framework 4.0 注意:我正在使用Visual Studio 2017 Enterprise 15.9.6,并且此代码针对.NET Framework 4.0
Option 1: tell MemoryStream
the correct number of bytes to use in the optional constructor overload; 选项1:告诉
MemoryStream
在可选的构造函数重载中使用的正确字节数; this will work for all serializers. 这将适用于所有序列化器。
Option 2, specifically for protobuf-net: use ProtoReader
; 选项2,专门用于protobuf-net:使用
ProtoReader
; the Deserialize
API accepts a Stream
or a ProtoReader
; Deserialize
API接受Stream
或 ProtoReader
; the latter can be constructed with a notional length and will not over-read 后者可以按名义长度构造,不会过度阅读
Option 3, again protobuf-net: use the *WithLengthPrefix
APIs for serialize and deserialize 选项3,还是protobuf-net:使用
*WithLengthPrefix
API进行序列化和反序列化
Note that any option assumes that your code can robustly determine the length, which should usually be communicated separately in whatever "framing" approach you're using. 请注意,任何选项均假定您的代码可以稳健地确定长度,通常应使用任何“成帧”方法将其分开传递。 Option 3 deals with this internally, but in many scenarios your own code still needs to be frame-aware so that you don't consume too much and read I to the next message (unless this is UDP with exactly one frame per packet, but... it isn't, because
MemoryMappedFile
). 选项3在内部处理此问题,但是在许多情况下,您自己的代码仍需要具有帧感知功能,这样您就不会消耗过多的代码并将我读到下一条消息(除非这是UDP,每个数据包恰好有一个帧,但是...并非如此,因为
MemoryMappedFile
)。 You also need to consider what happens if a payload is larger than the expected buffer size. 您还需要考虑如果有效负载大于预期的缓冲区大小会发生什么。
The simplest way of "framing" would be to reserve 4 bytes at the start of each logical block; 最简单的“成帧”方法是在每个逻辑块的开始处保留4个字节。 serialize starting at offset 4, and then write the number of bytes written back in at offsets 0-thru-3, using fixed 4-byte layout in your choice of endianness (usually "little").
从偏移量4开始进行序列化,然后在选择的字节顺序(通常为“小”)中使用固定的4字节布局,写回以偏移量0-thru-3写入的字节数。 When reading: read the first 4 bytes to get the length, then consume that- many bytes from the segment.
读取时:读取前4个字节以获取长度,然后从该段中消耗那么多字节。 This is essentially what
*WithLengthPrefix
does internally, except a few different layout options are supported. 这基本上是
*WithLengthPrefix
内部执行的操作,除了支持一些不同的布局选项。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.