How can I get elements from 3D matrix using specified indices in numpy?

Question

• I have a 3D matrix, in below example it's a (5, 4, 2) matrix: data_matrix
• I have a another index array of shape (5, 4) where each row of array represent the element location: indx_array

I don't know how can I get the required_output . I'm trying to arrange (1,2) elements of each row based on the indx_array

I don't want to use for loops!

data_matrix = np.array([
    [[0, 1], [2, 3], [4, 5], [6, 7]],
    [[8, 9], [10, 11], [12, 13], [14, 15]], 
    [[16, 17], [18, 19], [20, 21], [22, 23]], 
    [[24, 25], [26, 27], [28, 29], [30, 31]], 
    [[32, 33], [34, 35], [36, 37], [38, 39]]
])

indx_array = np.array([[3,2,1,0], [0,1,2,3], [1,0,3,2], [0,3,1,2], [1,2,3,0]])


# I want following result:
required_output = [
    [[6, 7], [4, 5], [2, 3], [0, 1]]
    [[8, 9], [10, 11], [12, 13], [14, 15]]
    [[18, 19], [16, 17], [22, 23], [20, 21]]
    [[24, 25], [30, 31], [26, 27], [28, 29]]
    [[34, 35], [36, 37], [38, 39], [32, 33]]
]

EDIT: Updated the indx_array to better illustrate the situation.

Answer 1

Can be done with a little bit of handing of the index array.

import numpy as np

_x = np.repeat(np.arange(indx_array.shape[0]),indx_array.shape[1])
_y = indx_array.ravel()

output = data_matrix[_x, _y].reshape(data_matrix.shape)

which results in the expected numpy array

array([[[ 6,  7],
        [ 4,  5],
        [ 2,  3],
        [ 0,  1]],

       [[ 8,  9],
        [10, 11],
        [12, 13],
        [14, 15]],

       [[18, 19],
        [16, 17],
        [22, 23],
        [20, 21]],

       [[24, 25],
        [30, 31],
        [26, 27],
        [28, 29]],

       [[34, 35],
        [36, 37],
        [38, 39],
        [32, 33]]])

Answer 2

• Numpy: Indexing
  • Numpy: Indexing Multi-dimensional arrays

In [637]: data_matrix.shape                                                                          
Out[637]: (5, 4, 2)
In [638]: indx_array.shape                                                                           
Out[638]: (5, 4)

You need advanced-indexing on the first 2 dimensions. The first dimension array needs to broadcast with the second (5,4). To do that I make a (5,1) arange :

In [639]: data_matrix[np.arange(5)[:,None], indx_array]                                              
Out[639]: 
array([[[ 6,  7],
        [ 4,  5],
        [ 2,  3],
        [ 0,  1]],

       [[ 8,  9],
        [10, 11],
        [12, 13],
        [14, 15]],

       [[18, 19],
        [16, 17],
        [22, 23],
        [20, 21]],

       [[24, 25],
        [30, 31],
        [26, 27],
        [28, 29]],

       [[34, 35],
        [36, 37],
        [38, 39],
        [32, 33]]])

Contrast my (5,1) index with the accepted _x (which is (5,4) ravelled):

In [640]: np.arange(5)[:,None]                                                                       
Out[640]: 
array([[0],
       [1],
       [2],
       [3],
       [4]])
In [641]: _x = np.repeat(np.arange(indx_array.shape[0]),indx_array.shape[1])                         
In [643]: _x                                                                                         
Out[643]: array([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4])

With broadcasting the _x doesn't need the repeat, (5,4); (5,1) is enough.

Broadcasting does a virtual repetition. This can be illustrated with the broadcast_to function:

In [648]: np.broadcast_to(np.arange(5)[:,None],(5,4))                                                
Out[648]: 
array([[0, 0, 0, 0],
       [1, 1, 1, 1],
       [2, 2, 2, 2],
       [3, 3, 3, 3],
       [4, 4, 4, 4]])
In [649]: _.strides                                                                                  
Out[649]: (8, 0)

It's that 0 strides that repeats without making copies. as_strided is the most useful stride_tricks function, especially when doing things like moving windows. Usually we just let the automatic broadcasting do the work without worrying too much about the how.