generator for iterating over the rows of a list of numpy arrays

Question

I'm having difficulties trying to come up with a generator that iterates over the rows of numpy arrays stored in a list.

I could accomplish this by using regular for loops:

list = [numpy.array([[0, 1, 2, 3, 4],
       [5, 6, 7, 8, 9]]), numpy.array([[0, -1, -2, -3, -4],
       [-5, -6, -7, -8, -9]])]

for array in list:
  for row in array:
    print(' '.join(map(str, row)))

but I would like to come up with a generator that does exactly that with less code. I tried the following, which works but has the number of array rows hardcoded:

mygen = (' '.join(map(str, i[j])) for j in range(2) for i in list)
for r in mygen:
  print (r)

Now when I try to change range(2) to range(i.shape[0]) I get the IndexError: tuple index out of range . Why this code doesn't work?

Answer 1

You can do this with itertools.chain.from_iterable :

>>> import itertools
>>> lst = [numpy.array([[0, 1, 2, 3, 4],
...        [5, 6, 7, 8, 9]]), numpy.array([[0, -1, -2, -3, -4],
...        [-5, -6, -7, -8, -9]])]
>>> for r in itertools.chain.from_iterable(lst):
...     print(' '.join(map(str, r)))
0 1 2 3 4
5 6 7 8 9
0 -1 -2 -3 -4
-5 -6 -7 -8 -9

Answer 2

Your attempt is just about right. The problem you are encountering is related to the precedence of the index inside the nested list comprehension. According to PEP-202 , you should use nested list comprehension as follows:

The form [... for x... for y...] nests, with the **last index
  varying fastest**, just like nested for loops.

Hence, if you change the ordering of the for loops inside the list comprehension then it works:

mygen = (' '.join(map(str, i[j])) for i in list for j in range(i.shape[0]) )
>>> list(mygen)
['0 1 2 3 4', '5 6 7 8 9', '0 -1 -2 -3 -4', '-5 -6 -7 -8 -9']