Saving values in a list of lists (Numpy)

Question

I have a list of lists containing values that I'd like to save for later use. Saving them to a CSV seems like a pain so first I would like to transfer the list into an array.

The name of the lists is 'modes'. Each entry in 'modes' contains a list of values representing resonant states for that mode. The first coordinate is the real part and the second is the imaginary part.

# The resonant states of the mode m = 1
In [178]: modes[1]
Out[178]: 
{0: array([ 1.12059699, -0.24524525]),
 1: array([ 2.71832424, -0.26526527]),
 2: array([ 4.28671194, -0.26526527]),
 3: array([ 5.87902919, -0.26526527]),
 4: array([ 7.44744745, -0.27266801]),
 5: array([ 9.02645242, -0.26526527]),
 6: array([ 10.61500625,  -0.28528529])}

There are around 100 modes each containing 50 coordinates. I want to transfer these points to an array with tuples like ( mode, real part, imag part). For instance, the above part would be:

array ([[ 1, 1.12059699, -0.24524525],
        [ 1, 2.71832424, -0.26526527],
        [ 1, 4.28671194, -0.26526527],
 etc....

This array would contain all resonant states of all modes. I would then take this array and store it into a CSV. Could someone help with this? If this isn't advisable I'm open to alternatives.

Answer 1

Something like this should work:

import numpy as np
temp = np.array(modes[0].values())
result = np.hstack( (np.zeros((temp.shape[0], 1)), temp) )
for i in range(1, len(modes)):
    temp = np.array(modes[i].values())
    result = np.vstack( (result, np.hstack((i * np.ones((temp.shape[0], 1)), temp))) )

Though I should mention that I put this together rather quickly. There ought to be nicer ways of writing the same thing that I'm not thinking of right now.

Answer 2

CSV is a challenging format with nested data. More so if you have nested numpy and native Python data structures.

If you have to read and write CSV, you will need to roll your own logic. Numpy's savetxt does not support the Python dict holding the numpy arrays you show in your example.

Python's pickle does support both native and numpy data as well as nested data:

m={0:{
    0: np.array([ 1.12059699, -0.24524525]),
    1: np.array([ 2.71832424, -0.26526527]),
    2: np.array([ 4.28671194, -0.26526527])},
   1:{
    3: np.array([ 5.87902919, -0.26526527]),
    4: np.array([ 7.44744745, -0.27266801]),
    5: np.array([ 9.02645242, -0.26526527]),
    6: np.array([ 10.61500625,  -0.28528529])}
    }

Given that data structure, you can save that as a pickled file thus:

with open(fn, 'w') as fout:
    fout.write(pickle.dumps(m))

And read it back:

with open(fn) as fin:
    mr=pickle.loads(fin.read())    
print mr

Prints:

{0: {0: array([ 1.12059699, -0.24524525]), 1: array([ 2.71832424, -0.26526527]), 2: array([ 4.28671194, -0.26526527])}, 1: {3: array([ 5.87902919, -0.26526527]), 4: array([ 7.44744745, -0.27266801]), 5: array([ 9.02645242, -0.26526527]), 6: array([ 10.61500625,  -0.28528529])}}