*(decimal*)d=XXXm results in another output than BinaryWriter.Write(XXXm)

Question

I'm writing an optimized binary reader/writer for learning purposes by myself. Everything works fine, until I wrote the tests for the en- and decoding of decimal s. My tests also include if the BinaryWriter of the .NET Framework produces compatible output to my BinaryWriter and vice versa.

I'm mostly using unsafe and pointers to write my variables into byte-arrays. Those are the results, when writing a decimal via pointers and via the BinaryWriter :

BinaryWriter....: E9 A8 94 23 9B CA 4E 44 63 C5 44 39 00 00 1A 00
unsafe *decimal=: 00 00 1A 00 63 C5 44 39 E9 A8 94 23 9B CA 4E 44

My code writing a decimal looks like this:

unsafe
{
    byte[] data = new byte[16];

    fixed (byte* pData = data)
        *(decimal*)pData = 177.237846528973465289734658334m;
}

And using BinaryWriter of .NET Framework looks like this:

using (MemoryStream ms = new MemoryStream())
{
    using (BinaryWriter writer = new BinaryWriter(ms))
        writer.Write(177.237846528973465289734658334m);

    ms.ToArray();
}

Microsoft made their BinaryWriter incompatible to the way decimal s are stored in memory. By looking into the referencesource we see that Microsoft uses an internal method called GetBytes, which means that the output of GetBytes is incompatible to the way decimals are stored in memory.

Is there a reason why Microsoft implemented writing decimal s in this way? May it be dangerous to use the way with unsafe to implement own binary formats or protocols because the internal layout of decimals may change in the future?

Using the unsafe way performs quite better than using GetBytes called by the BinaryWriter .

Answer 1

Microsoft itself tried to keep the decimal and the alignment of it's components as steady as possible. You can also see this in the mentioned referencesource of the .NET framework:

// NOTE: Do not change the order in which these fields are declared. The
// native methods in this class rely on this particular order.
private int flags;
private int hi;
private int lo;
private int mid;

Together with the usage of [StructLayout(LayoutKind.Sequential)] the structure gets aligned in exactly that way in the memory.

You get wrong results because of the GetBytes method using the variables which are building the data of the decimal internally not in the order they are aligned in the structure itself:

internal static void GetBytes(Decimal d, byte[] buffer)
{
    Contract.Requires((buffer != null && buffer.Length >= 16), "[GetBytes]buffer != null && buffer.Length >= 16");
    buffer[0] = (byte)d.lo;
    buffer[1] = (byte)(d.lo >> 8);
    buffer[2] = (byte)(d.lo >> 16);
    buffer[3] = (byte)(d.lo >> 24);

    buffer[4] = (byte)d.mid;
    buffer[5] = (byte)(d.mid >> 8);
    buffer[6] = (byte)(d.mid >> 16);
    buffer[7] = (byte)(d.mid >> 24);

    buffer[8] = (byte)d.hi;
    buffer[9] = (byte)(d.hi >> 8);
    buffer[10] = (byte)(d.hi >> 16);
    buffer[11] = (byte)(d.hi >> 24);

    buffer[12] = (byte)d.flags;
    buffer[13] = (byte)(d.flags >> 8);
    buffer[14] = (byte)(d.flags >> 16);
    buffer[15] = (byte)(d.flags >> 24);
}

It seems to me that the corresponding .NET developer tried to adapt the format presented by GetBytes to little endian, but made one mistake. He didn't only order the bytes of the components of the decimal but also the components itself. (flags, hi, lo, mid becomes lo, mid, hi, flags.) But little endian layout is only adapted to fields not to whole struct s - especially with [StructLayout(LayoutKind.Sequential)] .

My advice here is usually to use the methods Microsoft offers in their classes. So I would prefer any GetBytes or GetBits based way to serialize the data than doing it with unsafe because Microsoft will keep the compatibility to the BinaryWriter in any way. However, the comments are kinda serious and I wouldn't expect microsoft to break the .NET framework on this very basic level.

It's hard for me to believe that performance matters that strongly to favour the unsafe way over GetBits . After all we are talking about decimal s here. You still can push the int of GetBits via unsafe into your byte[] .