Read binary file of unknown size with mixed data types in python

Question

I need to read binary files which consist of 19 float32 numbers followed by a unknown number of uint32 numbers. How can I read such a file in python?

In Matlab the equivalent looks like this:

fid = fopen('myFile.bin','r');
params = fread(fid,19,'float');
data = fread(fid,'uint32');
fclose(fid);

Answer 1

Use numpy.fromfile() method and pass a file handle to it with the corresponding number of items to read.

import numpy as np
with open('myFile.bin', 'rb') as f:
    params = np.fromfile(f, dtype=np.float32, count=19)
    data = np.fromfile(f, dtype=np.int32, count=-1) # I *assumed* here your ints are 32-bit

Postpend .tolist() to closing paranthesis of fromfile() (like this: np.fromfile(...).tolist() ) if you want to get standard Python lists instead of numpy arrays.

Answer 2

For reading binary file I recomend using struct package

The solution can be written as follows:

import struct

f = open("myFile.bin", "rb")

floats_bytes = f.read(19 * 4)
# here I assume that we read exactly 19 floats without errors

# 19 floats in array
floats_array = struct.unpack("<19f", floats_bytes)

# convert to list beacause struct.unpack returns tuple
floats_array = list(floats_array)

# array of ints
ints_array = []

while True:
    int_bytes = r.read(4)

    # check if not eof
    if not int_bytes:
        break

    int_value = struct.unpack("<I", int_bytes)[0]

    ints_array.append(int_value)

f.close()

Note that I assumed your numbers are stored in little endian byte order, so I used "<" in format strings.