How to read a numpy ndarray from a block of memory?

Question

I have a block of memory that stores a 2D array of float32 numbers.

For example, the shape is (1000, 10) , and what I have in memory is something like a C array with 10000 elements.

Can I turn this into a numpy array just by specifying the shape and dtype ?

Answer 1

Turns out numpy supports interpreting a buffer as a 1-D array .

import numpy as np

def load_array(data, shape):
    return np.frombuffer(data, dtype=np.float32).reshape(shape)

Answer 2

Reading a memory-mapped array from disk involves numpy.memmap() function. The data type and the shape need to be specified again, as this information is not stored in the file.

Lets call the file containing data in disk : memmapped.dat

import numpy as np

array = np.memmap('memmapped.dat', dtype=np.float32,shape=(1000, 10))

Ref : https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.memmap.html and https://ipython-books.github.io/48-processing-large-numpy-arrays-with-memory-mapping/