我如何确保在编译时我的序列化函数创建具有预期大小的缓冲区？

Question

Currently we use packed structs with bitfields to create our datas

struct Data1
{
   uint8_t type;
   uint8_t value;
   bool aBool:1;
   uint8_t threeBits:3;
   uint8_t spare:4;
} __attribute__((packed))
const uint8_t Data1SerializedSize{3};
static_assert(sizeof(Data1) == Data1SerializedSize, "wrong Data1 size");

...

const uint8_t MaxBufferSize{32};

union AllDatas
{
   Data1 data1;
   ...
   uint8_t buffer[MaxBufferSize];
}__attribute__((packed))
static_assert(sizeof(AllDatas) <= MaxBufferSize);

It does work because we use the same cpu architecture everywhere and gcc is kind enough to allow us to use members of the AllDatas union that are not the one we last wrote.

void send(Serial& serial, const Data1& data)
{
    AllDatas u;
    u.data1 = data;
    serial.write(u.buffer, Data1SerializedSize);
}

But:

• It's undefined behavior if we follow the standard specification
• It's heavily platform dependant.
• It's rather messy and ineficient

But it offer the advantage to allow us to check at compile time that our structs have the size that we expect and hence, won't compile if that's not the case (work on 32 bits platform but not on a 64 one for example)

How can I create serialization functions that will ensure that at least, my output buffer has the size expected and that I did not forgot to add some members or on the contrary put one twice for example?

std::array<uint8_t, Data1SerializedSize> serialize(const Data1& data)
{
   std::array<uint8_t, Data1SerializedSize> buffer;
   uint8_t head{0};
   buffer[head] = data.type;
   ++head;
   buffer[head] = data.size;
   ++head;
   buffer[head] = (data.aBool & 0x1) 
                | ((data.threeBits << 1) & 0b1110) 
                | ((data.spare << 4) & 0b11110000);
   ++head;
   //static_assert(head == Data1SerializedSize);//won't work and above code looks more error prone
   return buffer;
}

The serialization format is clearly defined so I can't switch to a serialization library given that they can't (to my knowledge) be so tightly packed and so won't follow our format from the start.

As a side question, is there a way to avoid the double type declaration of the array (in the return type and in the function body)? I found no way to create a variable of the return type of the function in the function itself?

Answer 1

I finally managed to get something that can check at compile time not only that the size is right but that my serialization and deserialization functions work as expected as well

serializer.h

struct Data1Serializer
{
   static const uint8_t size{3};
   constexpr std::array<uint8_t, size> serialize(const Data1& data);
   constexpr Data1 deserialize(const std::array<uint8_t, size>& buffer);
}

serializer.cpp

constexpr auto
Data1Serializer::serialize(const Data1& data)
-> std::array<uint8_t, size>
{
   std::array<uint8_t, size> buffer;
   uint8_t head{0};
   buffer[head] = data.type;
   ++head;
   buffer[head] = data.value;
   ++head;
   buffer[head] = (data.aBool & 1)
            | ((data.threeBits >> 1) & 0b111)
            | ((data.spare >> 4) & 0b1111));
   ++head;
   assert(head == size);
   return buffer;
}

constexpr auto
Dat1Serializer::deserialize(const std::array<uint8_t, size>& buffer)
-> Data1
{
   Data1 data;
   data.type = buffer[0];
   data.value = buffer[1];
   data.aBool = buffer[2] & 1;
   data.threeBits = (buffer[2] >> 1) & 0b111;
   data.spare = (buffer[2] >> 4) & 0b1111;
   return data;
}

consteval bool checkData1()
{
   Data1 data{3,2,true,5,0};
   Data1Serializer serializer;
   const auto& result = serializer.deserialize(serializer.serialize(data));
   
   return (data.type == result.type)
       && (data.value == result.value)
       && (data.aBool == result.aBool)
       && (data.threeBits == result.threeBits)
       && (data.spare == result.spare);
}

static_assert(checkData1() == true, "Data1 serialization or deserialization failure");

With the out of bound checks of the array made at compile time, the assert to ensure that we filled the entire buffer and the comparison made in checkData1() I can be confident that my serialization and deserialization functions work.

I think that I will just move all that to unit tests to avoid the compilation time overhead and all the constexpr that are required for every function involved in the consteval check but it's nice to know that it's possible.