List Comprehension tuple over iterator in Python

Question

I'm learning about iterators and the groupby function in Python's itertools module. I understand that the following code takes strings of integers and prints tuples (x,y) where x is the number of times y repeats.

from itertools import groupby
s = input()
print(*[(len(list(c)), int(k)) for k, c in groupby(s)])

What I don't understand is "k, c in groupby(s)" as a part of a list comprehension. It would make sense to me if "groupby(s)" was a list of tuples but it's an iterator.

My question is how does a list comprehension produce a list from two variables over an iterator?

My testing has shown that I can use one variable over an iterator to produce a list. How is that list different?

Answer 1

The result of groupby(s) is not an iterator but tuples of keys and iterators. Passing two variables, "k,c" assigns k's to keys in groupby(s) and c to iterators in groupby(s).

There is ambiguity because

for k,c in groupby(s)

is the same as

for (k,c) in groupby(s)

where k and c are assigned to values within the tuples, and

for k in groupby(s)

assigns k to the tuples themselves.

Also, in the example code list(c) creates list with the iterator c.

Answer 2

The difference is that groupby(s) will return a series of pairs (tuples of length 2). You can bind each of these to a pair of variables.

As a general concept, this works just as if groupby(s) resolved to a list of the same pairs. The actual list comprehension can use any iterable at the right end; it's not restricted to a static sequence (list, tuple, string).

EXAMPLE

Let's look at a better example. groupby gathers adjacent elements with the same value. It does this with an iterator for each contiguous group of values. The sample string below has contiguous strings of B, C, D, and Z, and two separate strings of a single A. groupby will instantiate an iterator for each starting location; when invoked, the iterator will return a sequence of the consecutive letters, shown in the output.

from itertools import groupby
s = "BBCCCDDDDAZZA"
for k, c in groupby(s):
    print k, list(c)

Output:

B ['B', 'B']
C ['C', 'C', 'C']
D ['D', 'D', 'D', 'D']
A ['A']
Z ['Z', 'Z']
A ['A']

As you've noted, if we print merely the iterator c , all we get is the object handle. To obtain the letters, we need to invoke the iterator.

In the code you posted,

for k, c in groupby(s)

receives the series of (letter, iterator) pairs from groupby . The list function is what invokes the iterator, fetching the list of letters.

Does that clarify things at all?