How to read structured binary data from a file?

Question

The following C++ code writes a header to a file:

#include <iostream>

struct Header
{
    uint16_t name;
    uint8_t type;
    uint8_t padding;
    uint32_t width, height;
    uint32_t depth1, depth2;
    float dMin, dMax;
};

int main()
{

  Header header;
  header.name = *reinterpret_cast<const uint16_t*>("XO");
  header.type = true;
  header.width  = (uint32_t)512;
  header.height = (uint32_t)600;
  header.depth1  = (uint32_t)16;
  header.depth2 = (uint32_t)25;
  header.dMin = 5.0;
  header.dMax = 8.6;

  FILE* f = fopen("header.bin", "wb");
    fwrite(&header, sizeof(Header), 1, f);
}

I am looking to read these header.bin files using Python. In C++ I would be doing something like:

fread(&header, sizeof(Header), 1, f)

But I'm unsure how to read the bytes and convert them into the corresponding fields that the Header struct has in Python?

Answer 1

Use the struct module to define the binary layout of a C-like struct and de-/serialise it:

import struct

# Format String describing the data layout
layout = "H B x 2L 2L 2f"

# Object representing the layout, including size
header = struct.Struct(layout)

with open("header.bin", "rb") as in_stream:
    print(header.unpack(in_stream.read(header.size))

The layout is a format string describing the fields in-order, eg H for uint16_t , B for uint8_t , x for a pad byte, and so on.

Answer 2

I would do this with Python's ctypes , somewhat so you can share the Header header

Create a class from ctypes.Structure to map the types

import ctypes

class StructHeader(ctypes.Structure):
    _fields_ = [
        ("name", ctypes.c_uint16),
        ("type", ctypes.c_uint8),
        ...
    ]

And create a function which does what you want with a signature like

int header(struct Header &buffer)
{
    // open the file and write to buffer
    // opportunity for other features
}

Then you can compile a shared object to read it which returns that type

gcc -shared -Wl,-soname,your_soname \
    -o library_name file_list library_list

And call out with ctypes.CDLL to read the headers

header = ctypes.CDLL("mylib.so.1").header  # function named header
header.argtypes = [ctypes.POINTER(StructHeader)]
header.restype  = ctypes.c_int

# allocate struct for write
buffer = StructHeader()

# call out to function to write buffer
header(buffer)