State information and immutability of lists in Python

Question

I am a beginner in Python and using Mark Lutz's book to learn the fundamentals of Python.

Here's an example that the author uses to demonstrate storing state information using lists:

def tester(start):
    def nested(label):
        print(label,state[0])
        state[0] += 1
    state = [start]
    return nested

Here's the code to test state information:

F = tester(3)
F('sam')
F('sam')

You would see that the counter increases from 3 and then continues. In essence, above code stores initial state start (passed during initialization of the object) in [state] and increments it every time label is called.

However, I am unsure why Python doesn't throw an error in nested block. Specifically, [state] is local to tester and not nested.

To demonstrate what I mean, I am going to replace state[0] by state.

def tester(start):
    def nested(label):
        print(label,state) #Replaced state[0] with state
        state += 1         #Replaced state[0] with state
        print("after:",state)
    state = start          #Replaced state[0] with state
    return nested

Technically, above code should also work fine because all I have done is that replaced the list with the variable. However, PyCharm wouldn't even run this code. I get an error nboundLocalError: local variable 'state' referenced before assignment

Can someone please explain why the version with list works fine? The author has stated that "this leverages the mutability of lists, and relies on the fact that in-place object do not classify a name as local."

I am not really sure what that means. Can someone please help me? Thanks for any help extended to me.

Answer 1

You should read this section of the documentation .

Basically, in both versions the scope of the nested block allows it to interact with the namespace of the encompassing block. The difference is that you are not reassigning state in the first example, you're mutating it.

In the second example, Python knows that you are going to assign a value to that reference later in the function, so it is treated as a name from the local nested namespace, rather than the outer tester namespace.

You can use the nonlocal keyword to circumvent this and use the other reference from the other namespace

def tester(start):
    def nested(label):
        nonlocal state
        print(label,state) 
        state += 1         
        print("after:",state)
    state = start          
    return nested

Answer 2

From what I understand, because nested is nested under tester , it would have access to any objects and variables that belong to tester because tester is the parent function and nested is the child function in this case. Python will not produce an error because of inheritance .

And about replacing state[0] with state , Python automatically assumes that state is an integer because you are trying to add to it. While state is a list, and you can't add to is unless you append an element to it- which this isn't your case. The reason why state[0] works and not state is because state[0] is an element in the state list and it adds 0 to it.

Answer 3

It's a function of 1) how Python variable assignment actually just creates aliases (pointers) to underlying values in memory, and the difference between how mutable and immutable types are treated; and 2) some Python "magic" related to closures. The crux of your question is really the first point.

To address that, take for example the following:

a = 3
b = 3

Both a and b point to the same underlying object:

assert hex(id(a)) == hex(id(b)) 

is True. However, then setting b = 4 will cause b to point to a different object in memory (showing that the int is immutable).

However, a list is mutable (modifiable "in place"). For example: c = [2] will have the same memory location before and after an operation like c[0] = 3 .

There are a lot of implications of this very basic explanation that take some time to interpret. For example, variables can't “point to” other variables, but rather still point to underlying objects.

As a result, lists can exhibit "weird" behavior (another common, related confusion revolves around setting a default parameter value as a list that's then modified in the function), but can also be taken advantage of in the way your example shows.