I am trying to find a way to convert numpy array to hdf5 format

Question

I am trying to convert Numpy arrays that are 2D grids varying in time in a HDF5 format for several cases so for example the Numpy array has the following aspects: Case Number (0-100), Time (0-200years), X-grid point location (0-100m), y-grid point location (0-20m) plus the actual data point at this location (eg Saturation ranging from 0-100%). I am finding a bit difficult to efficiently store in HDF5 format. Its supposed to be used later to train an RNN model. I tried just assigning a Numpy to an HDF5 format (don't know if it worked as I didn't retrieve it). I was also confused about the different types of storage options for such a case and the best way to store it such that its easily retrievable to train a NN. I need to use HDF5 format as it seems to optimize the use/retrieval of large data as in the current case..I was also trying to find the best way to learn HDF5 format.. Thank you!

Answer 1

import h5py
import numpy as np

# Create a numpy array
arr = np.random.rand(3,3)

# Create a HDF5 file
with h5py.File('mydata.h5', 'w') as f:
    # Write the numpy array to the HDF5 file
    f.create_dataset('mydata', data=arr)

You can also use h5py library to append the data to existing hdf5 file instead of creating new one.