What happens to `for i in range()` when iterator is overwritten in nested while loop?

Question

What happens to for i in range() when iterator is overwritten in nested while loop? For example, why do the following snippets give different output? When I change the name of the variables i and j inside the while loop, the snippet behaves as I expected. However, when the while loop overwrites i and j , the for loop is affected. Is the resulting behavior of the for loop when its iterator is overwritten in the while predictable?

(a)

for i in range (0,3):
    for j in range (0,3):
         print "after nested for i,j",i,j
         counter = 0
         while counter < 3:
                counter += 1
                i = counter
                j = counter

has o/p:

after nested for i,j 0 0
after nested for i,j 3 1
after nested for i,j 3 2
after nested for i,j 1 0
after nested for i,j 3 1
after nested for i,j 3 2
after nested for i,j 2 0
after nested for i,j 3 1
after nested for i,j 3 2

(b) (the same code with the while commented)

for i in range (0,3):
    for j in range (0,3):
         print "after nested for i,j",i,j

has o/p

after nested for i,j 0 0
after nested for i,j 0 1
after nested for i,j 0 2
after nested for i,j 1 0
after nested for i,j 1 1
after nested for i,j 1 2
after nested for i,j 2 0
after nested for i,j 2 1
after nested for i,j 2 2

Answer 1

Your terminology is slightly wrong. In a for loop you have no access to an iterator. The iterator is kept hidden, behind the scenes. The following looping structures are equivalent.

for i in range(10):
    print(i)

it = iter(range(10)):
while True:
    try:
        i = next(it)
    except StopIteration:
        break
    print(i)

As you can see the iterator object ( it ) is kept hidden in the for loop. It's possible to expose the iterator in a for loop, but that's a different question.

What you are talking about is the name that the elements of the iterable are stored in. If you write over that name during the course of your loop, then that value will simply be ignored at the start of the next iteration of the loop. This is easy to see in the while version of the looping structure where the first thing that is done is that the name i is assigned the next element returned by the iterator.whil

I'm not sure of the purpose of your code, but it is possible to change the state of the iterator you are using. To do this you must write a coroutine . A coroutine is a specialised generator that is able to accept input.

def generator_range(start, end, step=1):
    "Simplified version of the range/xrange function written as a generator."
    counter = start
    while counter < end:
        yield counter 
        counter += step

def coroutine_range(start, end, step=1):
    "Special version of range that allows the internal counter to set."
    counter = start
    while counter < end:
        sent = yield counter 
        if sent is None:
            counter += step
        else:
            counter = sent

For simple range usages the generator version acts the same.

eg.

assert list(range(0, 10)) == [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
assert list(range(0, 10)) == list(generator_range(0, 10))
assert list(range(0, 10)) == list(coroutine_range(0, 10))

But we can do more complicated looping algorithms with the coroutine.

eg.

# skip numbers in range 3-7 inclusive
l = []
co = coroutine_range(0, 10)
item_to_send = None
while True:
    try:
        i = co.send(item_to_send)
        # if item_to_send is None then the above is the same as next(co)
        item_to_send = None
    except StopIteration:
        break
    if 3 <= i <= 7:
        item_to_send = 8
    else:
        l.append(i)

assert l == [0, 1, 2, 8, 9]

Answer 2

I think this is really a question of variable scope. i and j are in the local function namespace. While loops do not create new namespaces, so in the code fragment

     while len(list) < 2:
            i = i
            j = j

i and 'j' are still the same variables in the local namespace and all you did was reassign them to themselves. You didn't create new i or j . Its harmless, but those assignments should be removed.

Later, at the bottom of the while loop, when you do

            i = PixelCoord[Lightest[1]][0]
            j = PixelCoord[Lightest[1]][1] 

You reassign the i and j in the local function namespace that were holding the iterated values from the for loops to something else. The is relatively benign for j because you return to the inner for loop and j is reassigned the next iterated value. But it's a problem for x because it will hold the changed values until the outer for is reached again.

You can see the problem clearly with a couple of print statements

for i in range (1,array.shape[0]-1):
    print('outer for, i is', i)
    for j in range (1,array.shape[1]-1):
        print('inner for, i and j are', i, j)

The solution is to use different variable names.

Answer 3

If I am clear about your question, then here is solution:

Variables defined in a function have Function Scope and are only visible in the body of the function. You can use same name in different function.

Code 1:

def VAR1():
    var = 'foo'
    def inner():
        var = 'bar'
        print 'inside function, var is ', var
    inner()
    print 'outside function, var is ', var

VAR1()

Output:

inside function, var is  bar
outside function, var is  foo

But in a single function, variables are local. You can't use same name in different place

Code 1:

def VAR():
    var = 'foo'
    if True:
        var = 'bar'
        print 'inside if, var is ', var
    print 'outside if, var is ', var

VAR()

Output:

inside if, var is  bar
outside if, var is  bar

Read more Python's namespaces, scope resolution

Answer 4

This illustrates what is going on in your case(a) :

for i in range(0,2):
    for j in range(0,2):
        print(i,j)
        i = 'new-i'
        j = 'new-j'
        print('   ', i,j)

With output:

0 0                 # i and j set by the for statements
    new-i new-j     # i and j set by the inner assignments
new-i 1             # new j set by its for; i unchanged
    new-i new-j
1 0                 # new i and j set by for
    new-i new-j
new-i 1
    new-i new-j

There's nothing special about the i and j variables, except that they get new values at the start of their respective loops. And after the loops, i and j will have their last values within the loops, in this case new-i and new-j .

For simple loops like this, you can fiddle with the the value of i all you want, and it won't mess up the iteration (that is, the action of the for statement). But since it can confuse you and your algorithm (and your readers), it is generally not a good idea to reassign for iteration variables.

---

To fully separate your iteration variables from changes within the the while loop, you need to define a function like this:

def foo(i,j):
     print("after nested for i,j",i,j)
     counter = 0
     while counter < 3
         counter += 1
         i = counter
         j = counter

for i in range(3):
    for j in range(3):
        foo(i,j)

producing:

after nested for i,j 0 0
after nested for i,j 0 1
...
after nested for i,j 2 1
after nested for i,j 2 2

If on the other hand, you want changes in the inner while to control the iteration of i you need to use while in the outer loop as well:

i = 0
while i<3:
    j = 0
    while j<3:
        print("after nested for i,j",i,j)
        counter = 0
        while counter < 3:
            counter += 1
            i = counter
            j = counter

which only prints once ( i and j both jump to 3)