Why do these generator expressions behave differently?

Question

These two code fragments differ only in the way the list is constructed. One uses [] , the other list() .

This one consumes the iterable and then raises a StopIteration :

>>> try:
...     iterable = iter(range(4))
...     while True:
...         print([next(iterable) for _ in range(2)])
... except StopIteration:
...     pass
...
[0, 1]
[2, 3]

This one consumes the iterable and loops forever printing the empty list.

>>> try:
...     iterable = iter(range(4))
...     while True:
...         print(list(next(iterable) for _ in range(2)))
... except StopIteration:
...     pass
...
[0, 1]
[2, 3]
[]
[]
[]
etc.

What are the rules for this behaviour?

Answer 1

Refer to the PEP479 , which says that

The interaction of generators and StopIteration is currently somewhat surprising, and can conceal obscure bugs. An unexpected exception should not result in subtly altered behaviour, but should cause a noisy and easily-debugged traceback. Currently, StopIteration raised accidentally inside a generator function will be interpreted as the end of the iteration by the loop construct driving the generator .

(emphasis mine)

So the constructor of list iterates over the passed generator expression until the StopIteration error is raised (by calling next(iterable) without the second argument). Another example:

def f():
    raise StopIteration # explicitly

def g():
    return 'g'

print(list(x() for x in (g, f, g))) # ['g']
print([x() for x in (g, f, g)]) # `f` raises StopIteration

On the other hand, * comprehensions work differently as they propagate the StopIteration to the caller.

---

The behaviour that the linked PEP proposed is as follows

If a StopIteration is about to bubble out of a generator frame, it is replaced with RuntimeError , which causes the next() call (which invoked the generator) to fail, passing that exception out. From then on it's just like any old exception.

Python 3.5 added the generator_stop feature which can be enabled using

from __future__ import generator_stop

This behaviour will be default in Python 3.7.