Very strange behavior writing binary file

Question

I've run into an extremely perplexing issue when outputting binary data. I am writing a model converter that takes an ASCII file format and converts the values to binary. After parsing the file, I write the values out in binary. Everything works fine, up to a point, then things get weird.

The block of data that is being output looks like this:

struct vertex_t
{
    glm::vec2 texcoord;
    unsigned short weight_index_start;
    unsigned char weight_count;
};

Here is the binary block of data in question, with a human readable value following the # symbol.

00 00 80 3F # 1.000
AA AA AA 3E # 0.333
00 00       # 0
01          # 1

00 00 40 3F # 0.750
AA AA AA 3E # 0.333
01 00       # 1
01          # 1

00 00 40 3F # 0.750
00 00 00 00 # 0.000
02 00       # 2
01          # 1

...

Everything looks swell, up until the 11 th element, where something weird happens...

00 00 00 3F # 0.500
AA AA AA 3E # 0.333
09 00       # 9
01          # 1

FE FF 7F 39 # 2.4x10^-4
AA AA 2A 3F # 0.666
0D          # 13 (why is this here?!)
0A 00       # 10
01          # 1

00 00 40 3F # 0.75
00 00 80 3F # 1.0
0B 00       # 11
01          # 1

As you can see, a 0D is written in the middle of the struct for no apparent reason. Here is the relevant block of code that exports this struct:

for (const auto& vertex : mesh.vertices)
{
    ostream.write(reinterpret_cast<const char*>(&vertex.texcoord.x), sizeof(vertex.texcoord.x));
    ostream.write(reinterpret_cast<const char*>(&vertex.texcoord.y), sizeof(vertex.texcoord.y));
    ostream.write(reinterpret_cast<const char*>(&vertex.weight_index_start), sizeof(vertex.weight_index_start));
    ostream.write(reinterpret_cast<const char*>(&vertex.weight_count), sizeof(vertex.weight_count));
}

I have no idea how this could happen, but perhaps I'm missing something. Any help would be appreciated!

Answer 1

It seems like you are pushing carriage and newline into the file.

OD : Carriage return
0A : NL line feed, New line

See the ASCII table at http://www.asciitable.com/

Try opening a file with ios::binary paramater

example:

fstream output("myfile.data", ios::out | ios::binary);

Hope this helps!

Answer 2

You don't show how you're writing the data. Generally speaking, however, if the output format is not a text format, you must open the file in binary mode; off hand, it looks as though you're on a Windows system, and haven't done this. The binary value 0x0A corresponds to '\\n' , and if the file is not opened in binary mode, this will be converted to the system dependent new line indicator—under Windows (and probably most other non Unix systems), to the two byte sequence 0x0D, 0x0A.

Unless your output format is text, you must open the file in binary mode, both when reading and when writing. And (often forgotten) imbued with the "C" locale; otherwise, there may be code translation.

RE your update, with the write methodes: this does not output anything that you're guaranteed to be able to read later. (The need for a reinterpret_cast should tip you off to that.) If you're just spilling a too large data set to disk, to be reread later by the same process, fine (provided that the structures contain nothing but integral, floating point and enum types). In all other cases, you need to define a binary format (or use an already defined one, like XDR), format the output to it, and parse it on input. (Doing this in a totally portable fashion for floating point is decidedly non-trivial. On the other hand, most applications don't need total portability, and if the binary format is based on IEEE, and all of the targetted systems use IEEE, you can usually get away with interpreting the bit pattern of the floating point as an appropriately sized unsigned int, and reading and writing this.)