Only the last saved dataset exists when using h5py

Question

I am trying to save several datasets into a hdf5 file by h5py module, but it seems only the last one is saved. I think that because when a break statement was added, the first dataset is saved instead.

The code in problem is below. How can I fix it?

    set_num = 0
    for cur in data["init"]:
        '''
        got result as a list
        '''
        ipt = h5py.File(output_file, "w")
        s = str(set_num)
        ipt[s] = result
        '''
        create an attribute for ipt[s]
        '''
        set_num += 1
        ipt.close()
        #break

I apologize if there's any silly mistake.

Answer 1

You are opening, closing the file on each pass of the for: loop, and you have the attribute set to "w" , meaning that it is overwriting the existing file during each pass.

My recommendation is to instead open the file using the with clause, and nest the for: loop inside of that, which would make the intent clearer and obviate the need to explicitly close the file. This example might help (though it is not tested, so may need modifications):

with h5py.File(output_file, "w") as ipt:
    for set_num, curr in enumerate(data["init"]):
        s = str(set_num)
        ipt[s] = result

Answer 2

You only get the last dataset because you are opening the file in write mode ('w') , inside your loop. Simple solution is use append mode ( a') . Better, move the file open outside the loop and use the with...as: context manager.