shallow iteration with nditer

Question

I've got such an array:

>>>y = np.random.randint(0, 255, (2,2,3))
>>>array([[[242, 14, 211],
           [198,  7,   0]],

          [[235,  60,  81],
           [164,  64, 236]]])

And I have to iterate over each triplet element (unfortunately vectorization won't help me here...). So I tried:

for i, j in np.nditer(y):
print y[i, j],

hoping I'd get such an output:

[242, 14, 211], [198, 7, 0], [235, 60, 81], [164, 64, 236] , but no luck!

I get the error:

Traceback (most recent call last):

  File "<ipython-input-21-a336ef837a8a>", line 1, in <module>
    for i, j in np.nditer(y):    print y[i,j]

TypeError: iteration over a 0-d array

I'm quite sure I'm making a very obvious error... can anyone please help me?

Answer 1

Or reshape y

for i in y.reshape(-1,3):
    print i

A double iteration also works:

for x in y:
    for z in x:
        print z

Plain nditer iterates over each element of y ( nditer does not give you the indices):

for i in np.nditer(y):
    print i   
    # wrong y[i]

You'll need to dig more into the flags and documentation for nditer to iterate over 2 of its dimensions. While nditer gives access to the underlying iteration mechanism, you don't usually need to use it - unless you are doing something unusual, or trying to speed up code with cython .

---

Here's an example of getting 2 values from iteration on an nditer object. There is one value for each array in the op list. Both x and z are () shape arrays.

for x,z in np.nditer([y,y]):
    print x,z

There's more on the use of nditer at http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html

---

This doc page has an example using external_loop that dishes out the array in subarrays, rather than individually. I can accomplish the same with the 3d y by reordering its axes:

y3=y.swapaxes(2,0).copy(order='C')
for i in np.nditer(y3,order='F',flags=['external_loop']):
    print i,

[242  14 211] [198   7   0] [235  60  81] [164  64 236]

So we can use nditer to do this shallow iteration, but is it worth it?

---

In Iterating over first d axes of numpy array , I stumbled upon ndindex :

for i in np.ndindex(y.shape[:2]):
    print y[i],
# [242  14 211] [198   7   0] [235  60  81] [164  64 236]

ndindex uses nditer . The trick to generating shallow iteration is to use a subarray using just the dimensions you want to iterate over.

class ndindex(object):
    def __init__(self, *shape):
        ...
        x = as_strided(_nx.zeros(1), shape=shape, strides=_nx.zeros_like(shape))
        self._it = _nx.nditer(x, flags=['multi_index', 'zerosize_ok'], order='C')
    def __next__(self):
        next(self._it)
        return self._it.multi_index

Or stripping out the essential parts of ndindex I get:

xx = np.zeros(y.shape[:2])
it = np.nditer(xx,flags=['multi_index'])                               
while not it.finished:
    print y[it.multi_index],
    it.iternext()
# [242  14 211] [198   7   0] [235  60  81] [164  64 236]

Answer 2

It looks like you just need to flatten this down a level. You can use the chain operator from itertools.

from itertools import chain    


y = np.random.randint(0, 255, (2,2,3)
b = chain.from_iterable(y) # where b is a generator

list(b) output

[array([ 51, 119,  84]),
 array([ 50, 110, 193]),
 array([165, 157,  52]),
 array([239, 119,  83])]