Python Struct for Binary Data

Question

I'm trying to read binary files containing a stream of float and int16 values. Those values are stored alternating.

[float][int16][float][int16]... and so on

now I want to read this data file by a python program using the struct functions.

For reading a block of say one such float-int16-pairs I assume the format string would be "fh". Following output makes sense, the total size is 6 bytes

In [73]: struct.calcsize('fh')
Out[73]: 6

Now I'd like to read larger blocks at once to speed up the program...

In [74]: struct.calcsize('fhfh')
Out[74]: 14

Why is this not returning 12?

Answer 1

Quoting the documentation:

Note By default, the result of packing a given C struct includes pad bytes in order to maintain proper alignment for the C types involved; similarly, alignment is taken into account when unpacking. This behavior is chosen so that the bytes of a packed struct correspond exactly to the layout in memory of the corresponding C struct. To handle platform-independent data formats or omit implicit pad bytes, use standard size and alignment instead of native size and alignment: see Byte Order, Size, and Alignment for details.

https://docs.python.org/2/library/struct.html

If you want calcsize('fhfh') to be exactly twice calcsize('fh') , then you'll need to specify an alignment character.

Try '<fhfh' or '>fhfh' , instead.

Answer 2

You have to specify the byte order or Endianness as size and alignment are based off of that So if you try this:

>>> struct.calcsize('fhfh')
>>> 14
>>> struct.calcsize('>fhfh')
>>> 12

The reason why is because in struct not specifying an endian defaults to native

for more details check here: https://docs.python.org/3.0/library/struct.html#struct.calcsize