C++ serialization - use of reinterpret_cast from char * to a struct

Question

I am exchanging a struct called struct update_packet with other servers (of identical or similar system) running the same program through UDP socket using sendto(..) and recvfrom() .

update_packet needs to be in the general message format, which means its fields has predetermined fixed size and the size of the struct is the sum of the fields.

struct node {
    uint32_t IP;
    uint16_t port;
    int16_t nil;
    uint16_t server_id;
    uint16_t cost;
};

struct update_packet {
    uint16_t num_update_fields;
    uint16_t port;
    uint32_t IP;

    struct node * nodes;

    update_packet() :
        num_update_fields(num_nodes), IP(myIP), port(myport)
        {//fill in nodes array};
};

( update_packet contains a pointer array of struct node )

I used reinterpret_cast to send an instance of update packet via UDP, and the following compiles and sends to the correct destination.

int update_packet_size = sizeof(up);
sendto(s, reinterpret_cast<const char*>(&up), update_packet_size, 0,
       (struct sockaddr *)&dest_addr, sizeof(dest_addr));

However, when I receive it and try to decode it by

struct update_packet update_msg =
    reinterpret_cast<struct update_packet>(recved_msg);

I get an error

In function ‘int main(int, char**)’:
error: invalid cast from type ‘char*’ to type ‘update_packet’
           struct update_packet update_msg = 
           reinterpret_cast<struct update_packet>(recved_msg);

Why does this error occur, and how can I fix this?

Also, is this a correct way to exchange data in an instance of struct through sockets? If not, what should I do? Do I need a pack() ing function like in http://beej.us/guide/bgnet/examples/pack2.c ?

Answer 1

Generalities

The cast issue has been properly answered in other questions.

However, you should never rely on pointer cast for sending/receiving a struct through the network, for many reasons including:

• Packing : the compiler may align struct variables and insert padding bytes. This is compiler dependent, thus your code will not be portable. If the two communicating machines run your program compiled with different compilers, it will likely not work.
• Endianness : for the same reason, the byte order when sending a multibyte number (such as int) may be different between the two machines.

This would be resulting in a code which might work for some times, but a few years later which would cause a lot of problems, if someone changes the compiler, the platform, etc... As this is for an educational project you should try doing it the proper way...

For this reason, converting data from a struct into a char array for sending through the network or writing to file should be done carefully, variable by variable, and if possible taking endianness into account. This process is called "serializing".

Serialization in details

Serialization means you convert a data structure into an array of bytes, that can be sent over the network.

The serialized format is not necessarily binary : text or xml are possible options. If the amount of data is small, text is maybe the best solution, and you can rely on the STL only with stringstreams (std::istringstream and std::ostringstream)

There are several good libraries for serializing to binary, for instance Boost::serialization or QDataStream in Qt. You may also do it yourself, look SO for "C++ serializing"

Simple serializing to text using the STL

In your case, you might just serialize to a text string using something like:

std::ostringstream oss;

oss << up.port;
oss << up.IP;
oss << up.num_update_fields;
for(unsigned int i=0;i<up.num_update_fields;i++)
{
    oss << up.nodes[i].IP;
    oss << up.nodes[i].port;
    oss << up.nodes[i].nil;
    oss << up.nodes[i].server_id;
    oss << up.nodes[i].cost;
}

std::string str = oss.str();

char * data_to_send = str.data();
unsigned int num_bytes_to_send = str.size();

And for deserializing received data:

std::string str(data_received, num_bytes_received);
std::istringstream(str);


update_packet up;
iss >> up.port;
iss >> up.IP;
iss >> up.num_update_fields;
//maximum number of nodes should be checked here before doing memory allocation!
up.nodes = (nodes*)malloc(sizeof(node)*up.num_update_fields);
for(unsigned int i=0;i<up.num_update_fields;i++)
{
    iss >> up.nodes[i].IP;
    iss >> up.nodes[i].port;
    iss >> up.nodes[i].nil;
    iss >> up.nodes[i].server_id;
    iss >> up.nodes[i].cost;
}

This will be 100% portable and safe. You may verify data validity by checking the iss error flags.

Also you might, for safety:

• Use a std::vector instead of the node pointer. This will prevent memory leaks and other issues
• Check the number of nodes just after iss >> up.num_update_fields; , if it's too big just abort the decoding before allocating a huge buffer that will crash your program and maybe the system. Network attacks are based on "holes" like that : you may cause the a server to crash by making him allocate a buffer 100x larger than its RAM, if this kind of check is not made.
• If your networking API has a std::iostream interface, you may use directly the << and >> operators from it, without using the intermediate string and stringstreams
• You might think using space separated text is a waste of bandwidth. Think this only if your number of nodes is large, and makes the bandwidth use become non-negligible and critical. In that case, you need to serialize to binary. But don't do it if the text solution works perfectly (beware of premature optimization!)

Simple Binary serialization (not byte-order/endianness aware):

Replace:

oss.write << up.port;

By:

oss.write((const char *)&up.port, sizeof(up.port));

Endianness

But in your project, Big-Endian is required. If you are running on a PC (x86) you need to invert bytes in every field.

1)First option : by hand

const char * ptr = &up.port;
unsigned int s = sizeof(up.port);
for(unsigned int i=0; i<s; i++)
    oss.put(ptr[s-1-i]);

Ultimate code : detect endianness (this is not difficult to do - look for it on SO) and adapt your serialization code.

2)Second option : use a library like boost or Qt

These libraries let you choose the endianness of the output data. Then they auto-detect the platform endianness and do the job automatically.

Answer 2

You can't cast a pointer to a struct, but you can cast a pointer to a pointer to a struct.

Change

struct update_packet update_msg = 
       reinterpret_cast<struct update_packet>(recved_msg);

to

update_packet * update_msg = 
       reinterpret_cast<update_packet *>(recved_msg);

And yes, you need, at least pack() because the compiler on the sending side might add padding differently. However it is not 100% safe. You also have take into account that the sending and receiving machine differ in endianess. I would suggest that you look into proper serialization mechanisms.

Answer 3

You may also use:

struct update_packet update_msg;

memcpy(&update_msg, recved_msg, size-of-message);

You must however ensure that size-of-message is exactly what you are looking for.

Answer 4

Speaking just of decoding (your computer - your rules), both endianness and packing can be taken into account on GCC and Clang with a combo like this (it's using the Boost.Endian library):

#include <boost/endian/arithmetic.hpp>
using boost::endian::big_uint16_t;
using boost::endian::big_uint32_t;
using boost::endian::big_uint64_t;

#pragma pack(push, 1)

enum class e_message_type: uint8_t {
  hello = 'H',
  goodbye = 'G'
};

struct message_header {
    big_uint16_t size;
    e_message_type message_type;
    std::byte reserved;
};
static_assert(sizeof(header) == 4);

struct price_quote {
  big_uint64_t price;
  big_uint32_t size;
  big_uint32_t timestamp;
};
static_assert(sizeof(header) == 16);

template<class T> struct envelope {
  message_header header;
  T payload; 
};
static_assert(sizeof(envelope<price_quote>) == 20);

#pragma pack(pop)

// and then
auto& x = *static_cast<envelope const*>(buffer.data());