Lazy Python behavior with generators

Question

OK, right off the bat: I know Python generators are lazy. That said, this is blowing my mind; I don't get it. Pythonic friends, please don't crucify me for the ignorance of this question. This was not encountered in a real-world development situation, just a curiosity that has been bothering me.

def gen1(n):
    print('gen1')
    return 1

def gen2(n):
    print('gen2')
    return 2

print(gen2(gen1(0)))

This sample does exactly what I expect it to do, it prints gen1 then gen2 then 2.

The following code sample does not:

def gen1(n):
    print('gen1')
    yield 1

def gen2(n):
    print('gen2')
    yield 2

print(list(gen2(gen1(0))))

Instead, it just prints 'gen2' and [2]. So wait a hot sec, it calls gen2 first? It never evaluates gen1? WHAT?! Under the hood, how does it pull that devil magic off? What is actually taking place here? So the outermost thing is identified as a generator and it triggers some introspections that identifies that evaluation of the inner generator is unnecessary? What is a more informative way of tracing this behavior?

Answer 1

The code in the generator is not executed when the generator is created, but when it is consumed by a driving construct like a loop, a comprehension or another generator. The following might clear this up:

>>> def gen1(n):
...     print('gen1')
...     yield 1
...     print('gen1')
...     yield 2
... 
>>> g = gen1(5)  
# this returns the generator object, 
# but does not execute any code in it, hence 'lazy'
>>> for x in g:  
        # each 'x' assignment executes code til the next 'yield'
...     print x
... 
gen1
1
gen1
2

You might wonder how the program knows it is a genrator before the first yield statement, but the interpreter checks the entire function body first, and if there is any yield the function is a generator function whose body is not executed when called.