list comprehension with generator

Question

I used generator to generate some lists. However, it doesn't work like what I expected. I first used numgen1 to to generate the list while it doesn't work properly. And I switched to numgen2 which can gave me what I want properly. But both the numgen1 and numgen2 are basically the same (at least I think), why do they behave so differently? May anyone give me some explanations?

def numgen1(start, end, delta):
    curr=start 
    while curr[1] < end[1] or curr[2]<end[2]:
        yield curr
        curr[2] += delta


print 'Output1: ', [ i for i in numgen1([1,1,1],[1,1,5],1)]

def numgen2(start, end, delta):
    curr=start 
    while curr[1] < end[1] or curr[2]<end[2]:
        yield [curr[0], curr[1], curr[2]]
        curr[2] += delta


print 'Output2: ', [ i for i in numgen2([1,1,1],[1,1,5],1)]

Here are the outputs.

Output1:  [[1, 1, 5], [1, 1, 5], [1, 1, 5], [1, 1, 5]]
Output2:  [[1, 1, 1], [1, 1, 2], [1, 1, 3], [1, 1, 4]]

Follow up question:

After reading the answer from unutbu, I write one more generator that is used to test what unutbu said. But the generator doesn't behave as unutbu said. I am so confused about whether a generator is yielding a pointer or a copy of the value.

def numgen3(start, end, delta):
    curr=start 
    while curr<end:
        yield curr
        curr += delta

print list(numgen3(1,10,1))

Here is the output. [1, 2, 3, 4, 5, 6, 7, 8, 9]

This time I try to generate some numbers instead of some lists. But why aren't all the elements in the list 9? I didn't create a new number, I just yield the same number (curr). I expect the result of numgen3 should be similar to that of numgen1.

Answer 1

curr is a list. curr[2] += delta is modifying the list in-place .

When you yield curr , you are yielding the same list over and over. When you print Output1 you are seeing this same list being printed many times.

When you yield [curr[0], curr[1], curr[2]] you are generating a new list. Thus, when you print Output2 , you see different values.

---

Notice how modifying curr affects all items in result , because result is a list containing 3 items, each the same list curr :

curr = [0,0,0]
result = [curr for i in range(3)]
print(result)
# [[0, 0, 0], [0, 0, 0], [0, 0, 0]]

curr[2] = 100
print(result)
# [[0, 0, 100], [0, 0, 100], [0, 0, 100]]

---

You could "fix" numgen1 by yielding list(curr) since list(curr) returns a new list with the same elements as in curr (ie a "shallow copy"):

def numgen1(start, end, delta):
    curr=start 
    while curr[1] < end[1] or curr[2]<end[2]:
        yield list(curr)
        curr[2] += delta

print 'Output1: ', [ i for i in numgen1([1,1,1],[1,1,5],1)]

yields

Output1:  [[1, 1, 1], [1, 1, 2], [1, 1, 3], [1, 1, 4]]

---

Regarding numgen3 :

def numgen3(start, end, delta):
    curr=start 
    while curr<end:
        yield curr
        curr += delta

print list(numgen3(1,10,1))

It helps to make a mental distinction between mutable and immutable values. A list is mutable, numbers such as ints are immutable.

Lists are containers. You can mutate its contents without changing the reference to the container. Thus curr is a list, curr[2] += delta mutates the contents at index 2, but yield curr yields the very same list.

In numgen3 , curr is an immutable int . curr += delta assigns curr to a new immutable int . It no longer references the same object. yield curr yields that value. Those different values are accumulated in the list comprehension, and thus you see the result containing different values.

---

Here is another perspective on what it means to modify a list in-place : The modification is done in-place if the contents of the list change, while the memory address of the list itself does not change.

id(obj) returns the memory address of the object obj . Notice that modifying curr[2] does not change the id of curr :

In [162]: curr = [0,0,0]

In [163]: id(curr)
Out[163]: 196192940

In [164]: curr[2] += 1

In [165]: curr
Out[165]: [0, 0, 1]

In [166]: id(curr)
Out[166]: 196192940

Compare that with what happens when you increment a variable assigned to an int :

In [191]: curr = 1

In [192]: id(curr)
Out[192]: 150597808

In [193]: curr += 1

In [194]: id(curr)
Out[194]: 150597796

Here, curr is not being modified in-place. curr is simply being redirected to reference a new value at a new memory address.