Converting a list of lists into a 2D numpy array

Question

I have a list X which contains a list of lists of lists... eg

X = [ [[], [], [], []], [[], [] ,[], []] ]

When i try to convert this list above (X) into a numpy array using np.array() , I get the following:

array([list([[],[],[],[]]),
list([list([[],[],[],[]]),....) 

and the list goes on pun intended*.

What am I doing wrong here? I just need it in a normal array form such that I can index it in ways lists don't allow (ie array[:,1] )

Answer 1

If your lists are NOT of the same length (in each nested dimension) you CANT do a traditional conversion to a NumPy array because it's necessary for a NumPy array of 2D or above to have the same number of elements in its first dimension.

So you cant convert [[1,2],[3,4,5]] to a numpy array directly. Applying np.array will give you a 2 element numpy array where each element is a list object as - array([list([1, 2]), list([3, 4, 5])], dtype=object) . I believe this is the issue you are facing.

You cant create a 2D matrix for example that looks like -

[[1,2,3,?],
 [4,5,6,7]]

What you may need to do is pad the elements of each list of lists of lists to a fixed length (equal lengths for each dimension) before converting to a NumPy array.

I would recommend iterating over each of the lists of lists of lists as done in the code I have written below to flatten your data, then transforming it the way you want.

---

If your lists are of the same length, then should not be a problem with numpy version 1.18.5 or above.

a = [[[1,2],[3,4]],[[5,6],[7,8]]]
np.array(a)

array([[[1, 2],
        [3, 4]],

       [[5, 6],
        [7, 8]]])

However, if you are unable to still work with the list of list of lists, then you may need to iterate over each element first to flatten the list and then change it into a numpy array with the required shape as below -

a = [[[1,2],[3,4]],[[5,6],[7,8]]]
flat_a = [item for sublist in a for subsublist in sublist for item in subsublist]
np.array(flat_a).reshape(2,2,2)

array([[[1, 2],
        [3, 4]],

       [[5, 6],
        [7, 8]]])

Answer 2

Try this:

>>> import numpy as np
>>> a = np.array([[[1,2],[3,4],[5,6]],[[7,8],[9,10],[11,12]]])
>>> a
array([[[ 1,  2],
        [ 3,  4],
        [ 5,  6]],

       [[ 7,  8],
        [ 9, 10],
        [11, 12]]])
>>> a.reshape(4,-1)
array([[ 1,  2,  3],
       [ 4,  5,  6],
       [ 7,  8,  9],
       [10, 11, 12]])


>>> a.reshape(1,-1)
array([[ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12]])