How to convert [numpy.ndarray of ndarrays] to one ndarray of numbers

Question

I somehow got the following type of weird-looking, nested numpy array. I wonder if there's some way to convert it to a normal one (without all these 'array' in the output).

features
> array([array([1, 0]), array([2, 0]), array([3, 0]), array([4, 0]),
       array([5, 0]), array([6, 0]), array([7, 0]), array([8, 0]),
       array([9, 0])], dtype=object)

features.reshape((9,1))
> array([[array([1, 0])],
       [array([2, 0])],
       [array([3, 0])],
       [array([4, 0])],
       [array([5, 0])],
       [array([6, 0])],
       [array([7, 0])],
       [array([8, 0])],
       [array([9, 0])]], dtype=object)

features.flatten()
> array([array([1, 0]), array([2, 0]), array([3, 0]), array([4, 0]),
       array([5, 0]), array([6, 0]), array([7, 0]), array([8, 0]),
       array([9, 0])], dtype=object)

features.squeeze()
> array([array([1, 0]), array([2, 0]), array([3, 0]), array([4, 0]),
       array([5, 0]), array([6, 0]), array([7, 0]), array([8, 0]),
       array([9, 0])], dtype=object)

features.reshape((-1,2))
> ValueError: cannot reshape array of size 9 into shape (2)

This issue matters because it messes with the shape of the array. For example,

features.shape
> (9,)

while it should really be (9,2) .

I really appreciate your help.

Desired Output:

features
> array([[1, 0],
       [2, 0],
       [3, 0],
       [4, 0],
       [5, 0],
       [6, 0],
       [7, 0],
       [8, 0],
       [9, 0]], dtype=int32)

features.ndim
> 2

features.shape
> (9,2)

Update: @MaxNoe's comment pointed out my mistake that the array wasn't rectangular. However, after I corrected the mistake via debug terminal, I still cannot easily convert the array to dtype=int32. It seems that once numpy recognizes ndarray as object there's hardly a way back to treat it as numbers.

My interest is whether there's an easy way to make the conversion. If not, I'd better edit my code to avoid the situation from occurring.

To ensure that we have the same thing, you can run

import pickle
features = pickle.loads(b'\x80\x03cnumpy.core.multiarray\n_reconstruct\nq\x00cnumpy\nndarray\nq\x01K\x00\x85q\x02C\x01bq\x03\x87q\x04Rq\x05(K\x01K\x04K\x02\x86q\x06cnumpy\ndtype\nq\x07X\x02\x00\x00\x00O8q\x08K\x00K\x01\x87q\tRq\n(K\x03X\x01\x00\x00\x00|q\x0bNNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK?tq\x0cb\x89]q\r(h\x00h\x01K\x00\x85q\x0eh\x03\x87q\x0fRq\x10(K\x01K\x02\x85q\x11h\x07X\x02\x00\x00\x00i4q\x12K\x00K\x01\x87q\x13Rq\x14(K\x03X\x01\x00\x00\x00<q\x15NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\x00tq\x16b\x89C\x08\x01\x00\x00\x00\x00\x00\x00\x00q\x17tq\x18bh\x00h\x01K\x00\x85q\x19h\x03\x87q\x1aRq\x1b(K\x01K\x02\x85q\x1ch\x14\x89C\x08\x02\x00\x00\x00\x00\x00\x00\x00q\x1dtq\x1ebh\x00h\x01K\x00\x85q\x1fh\x03\x87q Rq!(K\x01K\x02\x85q"h\x14\x89C\x08\x03\x00\x00\x00\x00\x00\x00\x00q#tq$bh\x00h\x01K\x00\x85q%h\x03\x87q&Rq\'(K\x01K\x02\x85q(h\x14\x89C\x08\x04\x00\x00\x00\x00\x00\x00\x00q)tq*bh\x00h\x01K\x00\x85q+h\x03\x87q,Rq-(K\x01K\x02\x85q.h\x14\x89C\x08\x05\x00\x00\x00\x00\x00\x00\x00q/tq0bh\x00h\x01K\x00\x85q1h\x03\x87q2Rq3(K\x01K\x02\x85q4h\x14\x89C\x08\x06\x00\x00\x00\x00\x00\x00\x00q5tq6bh\x00h\x01K\x00\x85q7h\x03\x87q8Rq9(K\x01K\x02\x85q:h\x14\x89C\x08\x08\x00\x00\x00\x00\x00\x00\x00q;tq<bh\x00h\x01K\x00\x85q=h\x03\x87q>Rq?(K\x01K\x02\x85q@h\x14\x89C\x08\t\x00\x00\x00\x00\x00\x00\x00qAtqBbetqCb.')

Answer 1

as easy as that:

features = np.array([np.array([1, 0]), np.array([2, 0]), np.array([3, 0]), np.array([4, 0]),
   np.array([5, 0]), np.array([6, 0]), np.array([7, 0]), np.array([8, 0]),
   np.array([9, 0])], dtype=object)

features = np.reshape(features, (-1, 2)).astype('Int32')
print(features)
# output 
[[1 0]
 [2 0]
 [3 0]
 [4 0]
 [5 0]
 [6 0]
 [7 0]
 [8 0]
 [9 0]]

print(features.dtype) # int32
print(features.ndim)  # 2 
print(features.shape) # (9, 2)

Answer 2

One solution would be to use tolist and then convert back.

np.array(features.tolist(),dtype=np.int32)

I can only create your array in a weird way:

features = np.array([np.array([1, 0]), np.array([2, 0]), np.array([3, 0]), np.array([4, 0]),
                     np.array([5, 0]), np.array([6, 0]), np.array([7, 0]), np.array([8, 0]),
                     np.array([9])], dtype=object)
features[-1] = np.array([9,0])

>>>features

   array([array([1, 0]),
          array([2, 0]), array([3, 0]),
          array([4, 0]),
          array([5, 0]),
          array([6, 0]),
          array([7, 0]),
          array([8, 0]),
          array([9, 0])], dtype=object)

features = np.array(features.tolist(),dtype=np.int32)

>>>features
array([[1, 0],
       [2, 0],
       [3, 0],
       [4, 0],
       [5, 0],
       [6, 0],
       [7, 0],
       [8, 0],
       [9, 0]], dtype=int32)

Answer 3

I can recreate your array with:

In [126]: array=np.array                                                                               
In [127]: alist = [array([1, 0]), array([2, 0]), array([3, 0]), array([4, 0]), 
     ...:        array([5, 0]), array([6, 0]), array([7, 0]), array([8, 0]), 
     ...:        array([9, 0])]                                                                        
In [128]: len(alist)                                                                                   
Out[128]: 9
In [129]: arr = np.empty(9, dtype=object)                                                              
In [130]: arr[:]=alist                                                                                 
In [131]: arr                                                                                          
Out[131]: 
array([array([1, 0]), array([2, 0]), array([3, 0]), array([4, 0]),
       array([5, 0]), array([6, 0]), array([7, 0]), array([8, 0]),
       array([9, 0])], dtype=object)

np.stack is a good way of joining the arrays into one. np.vstack would also work.

In [132]: np.stack(arr)                                                                                
Out[132]: 
array([[1, 0],
       [2, 0],
       [3, 0],
       [4, 0],
       [5, 0],
       [6, 0],
       [7, 0],
       [8, 0],
       [9, 0]])

The reason I didn't simply copy-n-paste your original array is that it would skip the object dtype:

In [133]: np.array(alist)                                                                              
Out[133]: 
array([[1, 0],
       [2, 0],
       [3, 0],
       [4, 0],
       [5, 0],
       [6, 0],
       [7, 0],
       [8, 0],
       [9, 0]])

reshape , flattened etc do nothing because the object dtype shape is (9,), not anything that can be reshape into a (9,2). You have to take the object dtype seriously!