C++: How to read and write multi-byte integer values in a platform-independent way?

Question

I'm developing a simple protocol that is used to read/write integer values from/to a buffer. The vast majority of integers are below 128, but much larger values are possible, so I'm looking at some form of multi-byte encoding to store the values in a concise way.

What is the simplest and fastest way to read/write multi-byte values in a platform-independent (ie byte order agnostic) way?

Answer 1

XDR format might help you there. If I had to summarize it in one sentence, it's a kind of binary UTF-8 for integers.

Edit: As mentioned in my comment below, I "know" XDR because I use several XDR-related functions in my office job. Only after your comment I realized that the "packed XDR" format I use every day isn't even part of the official XDR docs, so I'll describe it seperately.

The idea is thus:

• inspect most-significant bit of byte.
  • If it is 0, that byte is the value.
  • if it is 1, the next three bits give "byte count", ie number of bytes in value.
    • mask out top nibble (flag bit plus byte count), concatenate the appropriate number of bytes and you've got the value.

I have no idea if this is a "real" format or my (former) coworker created this one himself (which is why I don't post code).

Answer 2

You might be interested in the following functions:

htonl, htons, ntohl, ntohs - convert values between host and network byte order

   uint32_t htonl(uint32_t hostlong);
   uint16_t htons(uint16_t hostshort);
   uint32_t ntohl(uint32_t netlong);
   uint16_t ntohs(uint16_t netshort);

man byteorder

Answer 3

Text would be my first choice. If you want a varying length binary encoding you have two basic choices:

• a length indication
• an end marker

You obviously make merge those with some value bits.

• For a length indication that would give you something where the length and some bits are given together (see for instance UTF-8),

• For an end marker, you can for instance state that MSB set indicates the last byte and thus have 7 data bits per byte.

Other variants are obviously possible.

Answer 4

You could try Network Byte Order

Answer 5

Google's protocol buffers provide a pre-made implementation that uses variable-width encodings.