Why is a set object stored as a frozenset and a list object as a tuple?

Question

I saw a blog post where it's mentioned "Use func.__code__.co_consts to check all the constants defined in the function" .

def func():
    return 1 in {1,2,3}
func.__code__.co_consts
(None, 1, ({1, 2, 3}))

Why did it return a frozenset ?

def func():
    return 1 in [1,2,3]
func.__code__.co_consts
(None, 1, )

Why did it return a tuple instead of a list? Every object returned from __code__.co_consts is immutable . Why are the mutable constants made immutable ? Why is the first element of the returned tuple always None ?

Answer 1

This is a result of the Python Peephole optimizer

Under "Optimizations", it says:

BUILD_LIST + COMPARE_OP(in/not in): convert list to tuple
BUILD_SET + COMPARE_OP(in/not in): convert set to frozenset 

See here for more information:

"Python uses peephole optimization of your code by either pre-calculating constant expressions or transforming certain data structures"

especially the part about " Membership Tests ":

"What Python for membership tests is to transform mutable data structures to its inmutable version. Lists get transformed into tuples and sets into frozensets."

Answer 2

All objects in co_consts are constants, ie they are immutable. You shouldn't be able to, eg, append to a list appearing as a literal in the source code and thereby modify the behaviour of the function.

The compiler usually represents list literals by listing all individual constants appearing in the list:

>>> def f():
...     a = [1, 2, 3]
...     return 1 in a
... 
>>> f.__code__.co_consts
(None, 1, 2, 3)

Looking at the byte code of this function we can see that the function builds a list at execution time each time the function is executed:

>>> dis.dis(f)
  2           0 LOAD_CONST               1 (1)
              2 LOAD_CONST               2 (2)
              4 LOAD_CONST               3 (3)
              6 BUILD_LIST               3
              8 STORE_FAST               0 (a)

  3          10 LOAD_CONST               1 (1)
             12 LOAD_FAST                0 (a)
             14 COMPARE_OP               6 (in)
             16 RETURN_VALUE

Creating a new list is required in general, because the function may modify or return the list defined by the literal, in which case it needs to operate on a new list object every time the funciton is executed.

In other contexts, creating a new list object is wasteful, though. For this reason, Python's peephole optimizer can replace the list with a tuple, or a set with a frozen_set , in certain situations where it is known to be safe. One such situation is when the list or set literal is only used in an expression of the form x [not] in <list_literal> . Another such situation is when a list literal is used in a for loop.

The peephole optimizer is very simple. It only looks at one expression at a time. For this reason, it can't detect that this optimization would be safe in my definition of f above, which is functionally equivalent to your example.