how to split a list of numpy arrays based on two conditions

Question

I have a list of including numpy array and want to rearrange it using two numbers. This is my list:

all_data=[np.array([[1., 1., 0.],[1., 1., 0.2],[1., 1., 0.1]]),\
          np.array([[2., 2., 0.1],[2., 2., 0.1],[2., 2., 0.1]]),\
          np.array([[3., 3., 0.2],[3., 3., 0.2],[3., 3., 0.2]]),\
          np.array([[1., 1., 100.],[1., 1., 110.],[1., 1., 110.]]),\
          np.array([[2., 2., 100.],[2., 2., 100.],[2., 2., 100.]]),\
          np.array([[3., 3., 100.],[3., 3., 110.],[3., 3., 120.]])]

These are my numbers:

n_iteration=3
n_layer=2

n_iteration gives me the number of iterations that I have and n_layer gives me the number data sets that I have in each iteration. It means I have two data sets in each iteration. At the moment, my all_data has 6 arrays. First three arrays are representing the data related to the first data set in three iterations. The last three arays are also the data for next data set in the three iteration. In other words, data sets are sorted out based on their iteration number but I want to sort iterations based on the data sets. I have written some simple data to make also a visual difference from my all_data to desired output. I want to have a list of sublists in which number of sublists is equal to n_iteration and numbers of arrays in each sublist equals n_layer :

[[np.array([[1., 1., 0.],[1., 1., 0.2],[1., 1., 0.1]]),\
  np.array([[1., 1., 100.],[1., 1., 110.],[1., 1., 110.]])],\
 [np.array([[2., 2., 0.1],[2., 2., 0.1],[2., 2., 0.1]]),\
  np.array([[2., 2., 100.],[2., 2., 100.],[2., 2., 100.]])],\
 [np.array([[3., 3., 0.2],[3., 3., 0.2],[3., 3., 0.2]]),\
  np.array([[3., 3., 100.],[3., 3., 110.],[3., 3., 120.]])]]

I tried the following code but it is giving me something else:

arranged_data=[]
for i in range (int (len(all_data)/n_iteration)):
    arranged_data.append([all_data[i], all_data[int (i+n_layer)]])

In advance, I do appreciate any help.

Answer 1

I don't know if I got the point, but maybe this is working as expected. Or not.

I'm using this data input, which is more readable:

all_data=[np.array([[1.],[2.],[3.]]),
          np.array([[4.],[4.],[4.]]),
          np.array([[5.],[5.],[5.]]),
          np.array([[6.],[7.],[7.]]),
          np.array([[8.],[8.],[8.]]),
          np.array([[9.],[10.],[11.]])]

First convert the list into a numpy array:

data_np = np.array(all_data)

Then, given the variables build an array with the indexes of the row you want to group:

layers = 3
iterations = 2
rows = np.array([np.array([i, i+layers]) for i in range(layers)])

Where rows results in

array([[0, 3],
       [1, 4],
       [2, 5]])

Now, use the numpy indexing:

res = data_np[rows]

Reshaping the res array, you should get your output (as a numpy array):

res.reshape(1, layers, iterations, layers)[0]
# array([[[ 1.,  2.,  3.],
#         [ 6.,  7.,  7.]],
# 
#        [[ 4.,  4.,  4.],
#         [ 8.,  8.,  8.]],
# 
#        [[ 5.,  5.,  5.],
#         [ 9., 10., 11.]]])