python dictionary comprehension in exec uses globals instead of locals

Question

I have found the following differences between using dictionary comprehensions and other methods in exec and eval. In summary the difference is that when using comprehensions the variables are taken from the globals argument but equivalent code not using comprehensions takes the variables from the locals argument.

This was found in Python 2.7.3 from the Python Software Foundation Windows installer.

When executing the following code:

locals1 = {"d":{1: 'x', 2: 'y', 3: 'z'}, "test":3}
globals1 = dict(globals().items() + [("d", {1: 'a', 2: 'b', 3: 'c'}), ("test", 2)])
exec "new = {key:d[key] for key in d if key != test}" in globals1, locals1
print locals1

The output is:

{'test': 3, 'new': {1: 'a', 3: 'c'}, 'd': {1: 'x', 2: 'y', 3: 'z'}}

Note that the dictionary (d) and the test value (test) are both taken from the globals argument.

When equivalent code is executed:

locals2 = {"d":{1: 'x', 2: 'y', 3: 'z'}, "test":3}
globals2 = dict(globals().items() + [("d", {1: 'a', 2: 'b', 3: 'c'}), ("test", 2)])
exec "new = d.copy(); new.pop(test)" in globals2, locals2
print locals2

This output is generated:

{'test': 3, 'new': {1: 'x', 2: 'y'}, 'd': {1: 'x', 2: 'y', 3: 'z'}}

In this case the dictionary (d) and the test value (test) are both taken from the locals argument.

A further indication is that execution of the code with the comprehension will fail with a variable not found exception if the dictionary and/or test value are not in the globals argument even if they are in the locals argument.

Please note that this is not a question about the use of exec and I have good reasons for using exec. The same situation can be demonstrated using eval.

Answer 1

This is entirely correct; a dictionary comprehension is executed as a function scope . Any variables referenced in that function scope not defined in that scope are presumed to be globals.

You get the same effect if you were to use an explicit function in your exec -ed code:

>>> locals3 = {"d":{1: 'x', 2: 'y', 3: 'z'}, "test":3}
>>> globals3 = dict(globals().items() + [("d", {1: 'a', 2: 'b', 3: 'c'}), ("test", 2)])
>>> exec "def f():\n    new = d.copy()\n    new.pop(test)\n    return new\nnew = f()" in globals3, locals3
>>> print locals3
{'test': 3, 'new': {1: 'a', 3: 'c'}, 'd': {1: 'x', 2: 'y', 3: 'z'}, 'f': <function f at 0x106bbcaa0>}

This is documented in the Displays for sets and dictionaries section:

Note that the comprehension is executed in a separate scope, so names assigned to in the target list don't “leak” in the enclosing scope.

In Python 2.x, list comprehensions do not get their own scope, something that has since been changed in Python 3.