python中的闭包是否捕获对象引用，或者仅捕获它们需要的属性？

Question

This question is about how intelligently python does its escape analysis.

Lets say I have the following program:

class Dog():
  breed = 'electronic dog'
  collar_type = 'microsoft'

sparky=Dog()
def get_dog_info():
  return sparky.breed

The function get_dog_info() clearly will have to close over sparky.breed. But, to do so, does the implementation also escape the entire Dog object? That is, would there be an additional memory cost to close over collar_type as well? Or is this a choice that is left up to the implementation?

Answer 1

Both Dog and sparky are referred to by the global namespace that forms your module, keeping both in memory.

If you were to run del Dog , sparky would still refer to the class (through it's __class__ reference) keeping it alive. The class refers to the two attributes that are part of it's definition, so they are kept alive as well. This is all independent of the get_dog_info function.

CPython keeps objects in memory based on reference counts; if anything in Python starts refering to an object somethere , that object's reference count is increased by 1, and decreased again when the reference is removed. When the count drops to 0, the object is removed from memory, and a garbage collection process breaks up circular references as needed to facilitate this process.

Note that because sparky is a global, the function code does not directly reference anything; globals are looked up at runtime. If you were to delete sparky too, all references would be cleared up. Because sparky in get_dog_info() is looked up in the global namespace, calling get_dog_info() would then result in a NameError .

If you did have a closure (reference to a variable in a parent function scope), the same rules would apply, except that the closure reference counts as another reference to the instance, thus indirectly to the class and the contained attributes.

So, considering the following example, where we do create a closure:

class Dog():
    breed = 'electronic dog'
    collar_type = 'microsoft'

def foo():
    sparky = Dog()
    def bar():
        return sparky.breed
    return bar

bar = foo()
del Dog

In the above example the Dog class remains in memory, because the bar closure still refers to an instance of that class:

>>> bar.__closure__
(<cell at 0x1012b2280: Dog object at 0x1012b5110>,)
>>> bar.__closure__[0].cell_contents
<__main__.Dog object at 0x1012b5110>
>>> bar()
'electronic dog'

Answer 2

Obviously in the code you've shown us there is no closure at all ( due to globals ). I assume it's just a snippet. Have a look at this code ( as an example ):

def test():
  class Dog():
    breed = 'electronic dog'
    collar_type = 'microsoft'

  sparky=Dog()
  def get_dog_info():
    return sparky.breed

  print get_dog_info.func_closure

test()

which shows that entire object sparky has been "closed" in get_dog_info . And indeed that has to be like that, because retrieving attribute of an object requires some knowledge about the object ( breed can be a property for example ). So there is no place to improve that.

Answer 3

As a supplement to Martijn's answer , I'll add the following about why a Dog object ( sparky ) is stored in the closure rather than a string ( sparky.breed ), which I think is at least part of your question.

This is because of the way the . operator works -- it accesses the breed attribute of sparky at the time of function invocation, and so the whole sparky object has to be stored. If you wanted to store only a string in the closure, you'd have to change the function code to refer directly to the string.

So in other words, given the following...

>>> class Dog():
...   breed = 'electronic dog'
...   collar_type = 'microsoft'
... 
>>> def get_dog_info_closure():
...     sparky = Dog()
...     def get_dog_info():
...         return sparky.breed
...     return get_dog_info
>>> get_dog_info = get_dog_info_closure()

...you can see that the function's closure contains a Dog object, rather than simply the string returned by sparky.breed :

>>> get_dog_info.func_closure
(<cell at 0x10049fa28: instance object at 0x1004a1cf8>,)
>>> get_dog_info.func_closure[0].cell_contents
<__main__.Dog instance at 0x1004a1cf8>

This means that you can retrieve the Dog object and modify it, and future invocations will reflect that modification:

>>> get_dog_info.func_closure[0].cell_contents.breed = ('actual '
                                                        'flesh-and-blood dog!')
>>> get_dog_info()
'actual flesh-and-blood dog!'

To store only the breed string, you'd have to refer to it separately:

>>> def get_dog_info_closure():
...     sparky = Dog()
...     sbreed = sparky.breed
...     def get_dog_info():
...         return sbreed
...     return get_dog_info
... 
>>> get_dog_info = get_dog_info_closure()
>>> get_dog_info.func_closure[0].cell_contents
'electronic dog'