beautiful for loop nested in the dict function : how does that work?

Question

I am currently reviewing this post about a sudoku solver in Python. I am trying to deconstruct it line by line, and I went through the following

digits   = '123456789'
rows     = 'ABCDEFGHI'
cols     = digits
#squares will give you all the squares of a Sudoku, from A1-A9 to I1-I9
squares  = cross(rows, cols)
unitlist = ([cross(rows, c) for c in cols] +
            [cross(r, cols) for r in rows] +
            [cross(rs, cs) for rs in ('ABC','DEF','GHI') for cs in ('123','456','789')])

#each square has 3 units : the row it is in, the column it is in, and the "block" of 3x3 squares it is in :

units = dict((s, [u for u in unitlist if s in u]) 
             for s in squares)

That last part is what dumbfounds me. We have, if I am not mistaken, a list comprehension, inside a generator. But what is the "for s in squares" doing inside the dict function ? How comes I can't write the following and obtain the same result ?

for s in squares :
    units3 = dict((s, [u for u in unitlist if s in u]))

This seems pretty obsure to me, unfortunately all my attempts at finding a ressource for this end up with results that point to basic tutorials on dict creation, or looping over dict values...

Could you please either tell me how a for loop inside a dict function works, or point me towards a good tuto on that topic ?

Answer 1

It's not a list comprehension, but a generator expression. The two are related, but no list is built. A generator expression produces a generator object, something that can be iterated over lazily and each step the loop expression is executed.

For example, you could create such an expression to calculate squares:

>>> squares = (i ** 2 for i in range(10))
>>> squares
<generator object <genexpr> at 0x10c832468>
>>> next(squares)
0
>>> next(squares)
1
>>> next(squares)
4

The next() function advances the iterator to produce the next value. In between, the generator is paused, the remaining values have yet to be computed.

In the example you found, the generator expression is the only argument to the dict() call, in which case the (...) for the generator expression can be omitted. You could also write dict((...)) and it'll produce the exact same results; if a call takes more than one argument those parentheses would be required instead. Nested inside the dict(...) generator expression is a list comprehension for just the dictionary value.

The generator produces (key, value) tuples, which the dict() callable uses to create a dictionary. See the dict() documentation :

[...] Otherwise, the positional argument must be an iterable object. Each item in the iterable must itself be an iterable with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value.

So the equivalent for loop statement would be:

units = {}
for s in squares:
    units[s] = [u for u in unitlist if s in u]

Note the [u for u in unitlist if s in u] list comprehension; it is a separate expression independent from the generator expression (but it does use the current value of s each time). The loop sets a value in a dictionary we had to create up front here, but otherwise each iterable step has the same outcome: a key s is set with the list comprehension result as the value.

In Python 2.7 and Python 3, instead of a generator expression, you can use a dictionary comprehension to produce the exact same dictionary:

units = {s: [u for u in unitlist if s in u] for s in squares}

The pattern for a dictionary comprehension is {<key expression>: <value expression> for ... in ... <optionally more if filters and for loops>} ; compare this to the dict((<key expression>, <value expression>) for ... in ... <optionally more if filters and for loops>) pattern used by the code you found. A dictionary comprehension is faster, because the interpreter no longer has to worry about how to stop and start a generator expression each step, nor does it have to locate the dict name and call out to it.