Python's struct.pack/unpack equivalence in C++

Question

I used struct.pack in Python to transform a data into serialized byte stream.

>>> import struct
>>> struct.pack('i', 1234)
'\xd2\x04\x00\x00'

What is the equivalence in C++?

Answer 1

There isn't one. C++ doesn't have built-in serialization.

You would have to write individual objects to a byte array/vector, and being careful about endianness (if you want your code to be portable).

Answer 2

You'll probably be better off in the long run using a third party library (eg Google Protocol Buffers), but if you insist on rolling your own, the C++ version of your example might be something like this:

#include <stdint.h>
#include <string.h>

int32_t myValueToPack = 1234;  // or whatever
uint8_t myByteArray[sizeof(myValueToPack)];
int32_t bigEndianValue = htonl(myValueToPack);  // convert the value to big-endian for cross-platform compatibility
memcpy(&myByteArray[0], &bigEndianValue, sizeof(bigEndianValue));
// At this point, myByteArray contains the "packed" data in network-endian (aka big-endian) format

The corresponding 'unpack' code would look like this:

// Assume at this point we have the packed array myByteArray, from before
int32_t bigEndianValue;
memcpy(&bigEndianValue, &myByteArray[0], sizeof(bigEndianValue));
int32_t theUnpackedValue = ntohl(bigEndianValue);

In real life you'd probably be packing more than one value, which is easy enough to do (by making the array size larger and calling htonl() and memcpy() in a loop -- don't forget to increase memcpy()'s first argument as you go, so that your second value doesn't overwrite the first value's location in the array, and so on).

You'd also probably want to pack (aka serialize) different data types as well. uint8_t's (aka chars) and booleans are simple enough as no endian-handling is necesary for them -- you can just copy each of them into the array verbatim as a single byte. uint16_t's you can convert to big-endian via htons(), and convert back to native-endian via ntohs(). Floating point values are a bit tricky, since there is no built-in htonf(), but you can roll your own that will work on IEEE754-compliant machines:

uint32_t htonf(float f)
{
   uint32_t x;
   memcpy(&x, &f, sizeof(float));
   return htonl(x);
}

.... and the corresponding ntohf() to unpack them:

float ntohf(uint32_t nf)
{
   float x;
   nf = ntohl(nf);
   memcpy(&x, &nf, sizeof(float));
   return x;
}

Lastly for strings you can just add the bytes of the string to the buffer (including the NUL terminator) via memcpy:

const char * s = "hello";
int slen = strlen(s);
memcpy(myByteArray, s, slen+1);  // +1 for the NUL byte

Answer 3

I was also looking for the same thing. Luckily I found https://github.com/mpapierski/struct

with a few additions you can add missing types into struct.hpp, I think it's the best so far.

To use it, just define you params like this

DEFINE_STRUCT(test,
    ((2, TYPE_UNSIGNED_INT))
    ((20, TYPE_CHAR))
    ((20, TYPE_CHAR))
)

The just call this function which will be generated at compilation

pack(unsigned int p1, unsigned int p2, const char * p3, const char * p4)

The number and type of parameters will depend on what you defined above. The return type is a char* which contains your packed data. There is also another unpack() function which you can use to read the buffer

Answer 4

https://github.com/karkason/cppystruct

#include "cppystruct.h"

// icmp_header can be any type that supports std::size and std::data and holds bytes
auto [type, code, checksum, p_id, sequence] = pystruct::unpack(PY_STRING("bbHHh"), icmp_header);

int leet = 1337;
auto runtimePacked = pystruct::pack(PY_STRING(">2i10s"), leet, 20, "String!");
// runtimePacked is an std::array filled with "\x00\x00\x059\x00\x00\x00\x10String!\x00\x00\x00"
// The format is "compiled" and has zero overhead in runtime

constexpr auto packed = pystruct::pack(PY_STRING("<2i10s"), 10, 20, "String!");
// packed is an std::array filled with "\x00\x01\x00\x00\x10\x00\x00\x00String!\x00\x00\x00"

Answer 5

您可以查看Boost.Serialization ，但我怀疑您是否可以使用与Python的包相同的格式。

Answer 6

You can use union to get different view into the same memory.

For example:

union Pack{
   int i;
   char c[sizeof(int)];
};

Pack p = {};
p.i = 1234;
std::string packed(p.c, sizeof(int)); // "\xd2\x04\x00\0"

As mentioned in the other answers, you have to notice the endianness.