Iterating over a list of arrays while making changes to each array

Question

This is a follow-up post from a previous question of mine: Referring to arrays in a for-loop .

I would like to generalize the solution proposed there in order to be able to complete more complex tasks, such as attaching a column to each array that contains the result of some calculation:

import numpy as np
list=[one, two, three]

for arr in list:
    arr=np.column_stack([arr,5*arr[:,2]-arr[:,1])])

All three arrays have the same dimensions.

Answer 1

In [59]: one=np.arange(6).reshape(2,3)
In [60]: two=np.arange(6).reshape(2,3)

Forget about the loop for the moment, and just try to change one :

In [61]: arr = one
In [62]: arr=np.column_stack([arr,5*arr[:,2]-arr[:,1]])
In [63]: arr
Out[63]: 
array([[ 0,  1,  2,  9],
       [ 3,  4,  5, 21]])
In [65]: one
Out[65]: 
array([[0, 1, 2],
       [3, 4, 5]])

This action has changed arr , but not one . Originally arr referenced the same object ( ndarray ) as one , but after the new assignment, it referenced a new array.

In

for arr in alist:
    arr = ....

arr is assigned an element of alist . But then in the loop it is assigned another something else, without changing the original object. On the next iteration, arr is assigned the next element in the list, and so on.

You need to keep in mind several things.

• how Python assigns values to variables
• how Python assigns values to an iteration variable
• what functions like column_stack to

In your previous question

In [69]: for arg in [one,two]:
    ...:     arg[:,1:] += 10
    ...:     
In [70]: one
Out[70]: 
array([[ 0, 11, 12],
       [ 3, 14, 15]])
In [71]: two
Out[71]: 
array([[ 0, 11, 12],
       [ 3, 14, 15]])

this works because the arg[:,1:] += 10 is modifying the array currently assigned to arg . An array is mutable ; element values can be changed in-place.

np.column_stack() does not act in-place. It makes new array.

About the only way that you can change one and two with a list is a sequence of operations like:

In [72]: newlist=[np.column_stack([arr,5*arr[:,2]-arr[:,1]]) for arg in [one,two]]
In [73]: newlist
Out[73]: 
[array([[ 0,  1,  2,  9,  9],
        [ 3,  4,  5, 21, 21]]), array([[ 0,  1,  2,  9,  9],
        [ 3,  4,  5, 21, 21]])]
In [74]: one
Out[74]: 
array([[ 0, 11, 12],
       [ 3, 14, 15]])
In [75]: one, two = newlist
In [76]: one
Out[76]: 
array([[ 0,  1,  2,  9,  9],
       [ 3,  4,  5, 21, 21]])

In[72] creates a new list, with new arrays. In[75] assigns these new arrays to the variables one and two . This wipes out their previous references. In effect I did one=[np.column_stack([one,5*one[:,2]-one[:,1]]) , and similarly for two .

Answer 2

Assigning to the variable doesn't update the list, it's just a temporary reference to the value that's in the array.

Use enumerate to get the list index, then you can replace it with the result.

for index, arr in enumerate(myList):
    myList[index] = np.column_stack([arr,5*arr[:,2]-arr[:,1])])

Also, avoid using the names of built-in classes and functions as your own variable names. list is a standard class.

Answer 3

Don't forget that in Python, variables are references to values. The iteration variable is a variable, so assigning a new value to this variable do not change the iterable.

In this case, why not use lists comprehensions ?

import numpy as np
my_list = [one, two, three]

my_list = [np.column_stack([arr,5*arr[:,2]-arr[:,1])]) for arr in list]

And please, don't shadow list type…

Answer 4

I'm afraid the solution you are referencing can only be generalized to in-place changes to the array. Anything that changes the size will AFAIK create a new array.

So this

>>> X,Y,Z = (np.arange(i+1, i+10, dtype=float).reshape(3, 3) for i in range(3))
>>> L = [X,Y,Z]
>>> for arr in L:
...    np.sin(arr, out=arr)

or this

>>> for arr in L:
...     arr[1] = arr[1, ::-1]

will work.

You can even replace the entire array, as long as you do it in-place:

>>> for arr in L:
...     arr[...] = 1.7

But this

>>> for arr in L:
...    np.append(arr, arr[-1])

will not change the original arrays but create new ones. Assigning back to arr won't help because it just rebinds the name arr to the new object.