Alternatives to fread and fwrite for use with structured data

Question

The book Beginning Linux Programming (3rd ed) says

"Note that fread and fwrite are not recommended for use with structured data. Part of the problem is that files written with fwrite are potentially nonportable between different machines."

What does that mean exactly? What calls should I use if I want to write a portable structured data reader or writer? Direct system calls?

Answer 1

The book is wisely cautioning against reading an block of bytes from a file directly into a data structure.

The problem with this is that there can be unnamed padding bytes between individual elements of a data structure, and the number and position of these bytes is entirely implementation dependent.

You can still use the fread and fwrite calls to read and write data from and to a file, but you should read and write each element of the data structure individually, rather than reading or writing the whole struct at once.

There are other portability concerns you'll want to keep in mind as well. For example, the various numeric types have implementation-dependent sizes. For portability, you can use the types defined in the stdint.h header.

There may also be differences in floating point and unsigned integer representation, but most systems and file formats now use IEEE 754 and two's-complement, respectively, so compatibility issues are far less frequent with those types. Just make sure you know what your specifications say.

Answer 2

Data serialization is topic of your interest.

It's about sizes of variables, it's about coding (strings might be utf-8, utf16... etc), it's about endianess (BigEndian, LowEndian).

For portable solution I would recommend you to take a look at Google ProtocolBuffers and Thrift .

Answer 3

If portability of the data is a concern for you, you should look into Serialization techniques and libraries, in particular s11n JSON YAML XDR ASN1 Jansson XML etc.

Ask yourself about your data and your application in a couple of years?...

Textual representations are generally less "brittle" than binary ones.