Reshaping numpy array to be similar to different array with arbitrarily nested subarrays

Question

I have two arrays a and b . If I run a.shape I get (10,) and if I run (b.shape) I get (10,) however each of those values are arbitrarily nested arrays which are not equal in shape to the others in itself or to the corresponding value in the other array. For example a[0].shape returns (122,) and b[0].shape returns (3900,) and b[5].shape returns (5200,64) so they aren't even consistent within the same array.

I know that they have the same total number of elements using the recursive solution found at Iterating through a multidimensional array in Python (no standard numpy functions seemed to be able to drill as deeply as necessary into the arrays):

def iterThrough(lists):
  if not hasattr(lists[0], '__iter__'):
    for val in lists:
      yield val
  else:
    for l in lists:
      for val in iterThrough(l):
        yield val
            si = 0

for value in iterThrough(a): #and b
    si += 1

Both return 3066752 . I understand that this is quite messy, but I need it to be in that shape to compute a mathematical function later on and I'm guessing it's easier to to do this than to rewrite that equation to reflect a nicely formatted array.

How can I make array a exactly identical in all its nested shapes to array b ?

Answer 1

Here's a solution that does the job, but is guaranteed to be slow, because it is recursive, iterative, and vectorizes nothing at all:

import copy

# Generator function:
# Returns successive scalars from any crazily nested array
def arr_values(arr):
    for elem in arr:
        if (isinstance(elem, np.ndarray)):
            for sub_arr_elem in arr_values(elem):
                yield sub_arr_elem
        else:
            yield elem

# Generator function:
# Returns successive tuples (vector, index) from any arbitrarily nested array,
# such that assigning a value to vector[index] is the same as assigning the value
# to a position in the input array.
def arr_positions(arr):
    for pos,elem in enumerate(arr):
        if (isinstance(elem, np.ndarray)):
            for inner_arr_pos in arr_positions(elem):
                yield inner_arr_pos
        else:
            yield arr, pos

# Create a new array, having the shape of `b` (and incidentally, b's data also)
arr_in_shape_of_b = copy.deepcopy (b)

# Get the iterator for successive assignable positions in
# our destination array
iter_dest = arr_positions(arr_in_shape_of_b)

# Get the iterator for successive scalars from our source array
iter_src = arr_values(a)

# Now, iterate over the two iterators in tandem, and
# perform assignment of successive values from source,
# into successive positions in destination.
for ((dest_vec, pos), val) in zip(iter_dest, iter_src):
    dest_vec[pos] = val

Testing it out:

I tested with this demo data for a , and b :

a = np.array ([np.arange(6).reshape(2,3), np.array([np.arange(8).astype(np.ndarray),
                                                    23], dtype=np.ndarray),24], dtype=np.ndarray)
print (a.shape)
print (a)

b = np.roll(-1 * a, 2, axis=0)
print (b.shape)
print (b)

So, the input arrays a and b look like this:

(3,)
[array([[0, 1, 2],
       [3, 4, 5]])
 array([array([0, 1, 2, 3, 4, 5, 6, 7], dtype=object), 23], dtype=object)
 24]
(3,)
[array([array([0, -1, -2, -3, -4, -5, -6, -7], dtype=object), -23],
      dtype=object)
 -24 array([[ 0, -1, -2],
       [-3, -4, -5]])]

The output looks like this:

(3,)
[array([array([0, 1, 2, 3, 4, 5, 0, 1], dtype=object), 2], dtype=object) 3
 array([[ 4,  5,  6],
       [ 7, 23, 24]])]