Python生成器与列表理解冲突

Question

I've been messing around in Python with generator functions. I want to write a function that took a generator whose values were tuples, and returns a list of generators, where each generator's values correspond to one index in the original tuple.

Currently, I have a function which accomplishes this for a hardcoded number of elements in the tuple. Here is my code:

import itertools

def tee_pieces(generator):
    copies = itertools.tee(generator)
    dropped_copies = [(x[0] for x in copies[0]), (x[1] for x in copies[1])]
    # dropped_copies = [(x[i] for x in copies[i]) for i in range(2)]
    return dropped_copies

def gen_words():
    for i in "Hello, my name is Fred!".split():
        yield i

def split_words(words):
    for word in words:
        yield (word[:len(word)//2], word[len(word)//2:])

def print_words(words):
    for word in words:
        print(word)

init_words = gen_words()
right_left_words = split_words(init_words)
left_words, right_words = tee_pieces(right_left_words)
print("Left halves:")
print_words(left_words)
print("Right halves:")
print_words(right_words)

This correctly splits the generator, leading to left_words containing the left halves and right_words containing the right halves.

The problem comes when I try to parameterize the number of generators to be created, using the commented out line above. As far as I know it should be equivalent, but when I use that line instead, both left_words and right_words end up containg the right half of the word, giving an output like this:

Left halves:
lo,
y
me
s
ed!
Right halves:
lo,
y
me
s
ed!

Why is this happening? How can I accommplish the desired result, namely parameterize the number of pieces to split the generator into?

Answer 1

This has to do with python's lexical scoping rules. The classical "surprising" example for demonstrating it:

funcs = [ lambda: i for i in range(3) ]
print(funcs[0]())
=> 2  #??
print(funcs[1]())
=> 2  #??
print(funcs[2]())
=> 2

Your examples is another result of the same rules.

To fix, you can "break" the scoping with an additional function:

def make_gen(i):
    return (x[i] for x in copies[i])
dropped_copies = [make_gen(i) for i in range(2)]

This binds the the value of i to the specific value passed to a specific call to make_gen , which achieves the desired behavior. Without it, it is bound the "the current value of the variable named i", which ends up as the same value for all generators you create (as there's only one variable named i ).

Answer 2

Too add to shx2's answer, you could also substitute the additional function by a lambda:

dropped_copies = [(lambda j: (x[j] for x in copies[j]))(i) for i in range(2)]

This too creates a new scope when the lambda gets called, as is abundantly clear by the different variable name. It would however also work with using the same name, since the parameter inside the lambda shadows the one inside the generator:

dropped_copies = [(lambda i: (x[i] for x in copies[i]))(i) for i in range(2)]

---

This sort of scoping seems very confusing but becomes more intuitive if you rewrite the generator as a for loop:

dropped_copies = []
for i in range(2):
    dropped_copies.append((x[i] for x in copies[i]))

_{Note that this is broken in the same way the original list comprehension version is.}

Answer 3

This is because dropped_copies is a pair of iterators, and when the iterators are evaluated, i has already been incremented to 1.

Try use list comprehension, you can see the difference:

dropped_copies = [[x[i] for x in copies[i]] for i in range(2)]