python如何执行变量分配？

Question

This is not so much a "how-to" question, as a question about how python functions to store variables.

As far as I understand (please correct me if i am wrong), when a variable is created in (say) C, the value is stored in memory and the address of that memory is stored under the variable name. In python it creates the value in memory, and then "tags" that memory address with the variable name. So that if you do this:

>>> x = 3
>>> y = x
>>> id(x)
42
>>> id(y)
42

the memory addresses are the same for x and y

then if i do this:

>>> x = 4
>>> print y
3

as it should. If i query the addresses:

>>> id(x)
46
>>> id(y)
42

it appears that python has created new memory for the new x value and y remains the same. but then when i do this:

>>> y = 4
>>> id(y)
46

it seems that python has searched through the memory looking for a value that matches the input variable, and then tagged that memory address with the name "y" - if no such value is found then it creates a new value in memory and then tags that

am i right in my thinking here?

the problem that occurs to me that this is all well and good for my little "x" and "y" example, but what about a program with millions of variables? does this mean that whenever a variable value is assigned, it has to search through the whole memory, looking for a match, before assigning new memory?

Intuitively, while this approach would definitely save a lot of space, it would probably not be very time efficient, however I imagine that a lot of people, much smarter than me, have put a lot of effort and work into making python, so obviously this isnt the case. So my question is what am i missing in my understanding that makes this method a good approach for memory management?

Is it simply a matter that searching through the list is at most time complexity O(n), and so with fast processors, this is not really an issue? or is there something deeper?

Answer 1

First off, what you are asking is literally defined as " implementation defined ".

My guess is that Python is actually doing that check at compile-time, optimizing the memory layout given what it knows.

I can almost guarantee you it does NOT search through every variable. First, it would have to know what's mutable and what's not (hint: "what's not" is a very short list). If it's mutable, it can't just re-use the old one.

Answer 2

There's a detailed description for the CPython implementation here:

http://www.laurentluce.com/posts/python-integer-objects-implementation/

The behaviour you are seeing is because python special-cases the handling of small integers, as they are quite common. Try this, for instance, to see a different result.

>>> x = 88888888888888888888888
>>> y = 88888888888888888888888
>>> id(x)
4321062304
>>> id(y)
4321062344

Answer 3

variable is created in (say) C, the value is stored in memory and the address of that memory is stored under the variable name

No. In C, the variable is created, and obtains a location in memory no later than the execution of the statement which declares it (in so far as declarations are executable). Assignment to that variable alters its memory location.

In python it creates the value in memory, and then "tags" that memory address with the variable name.

Right. Or, equivalently, the variable holds a pointer (of whatever sort) to the object.

id

id does NOT return memory addresses. It returns an integer which is only returned for that specific object, during the life of that object.

seems that python has searched through the memory looking for a value that matches the input variable

This behaviour is implementation defined, and in CPython happens only for integers 0-127. It doesn't happen in the case of any other value. You may also be able to "defeat" this interning behaviour.

In addition, there is no scanning. Interning likely uses an array to store the interned values, because these are integers.

Answer 4

I am wary of saying that id represents a "memory address" - it happens to be implemented in terms of memory addresses in current versions of CPython, but this is hardly guaranteed by anything (and it is different in, for example, PyPy).

But other than that quibble, your thinking is roughly right, although the mechanism is a little simpler and quicker than you imagine.

What CPython (but not necessarily other implementations) does instead is to preallocate and cache a certain amount of "small" ints (currently everything from -5 up to and including 256). Then when one of them is requested, it finds it in this cache, instead of allocating a new Python object. But instead of the O(n) linear search you imagine, it is actually implemented as an O(1) C array lookup. You can see how it works in CPython's intobject.c :

#ifndef NSMALLPOSINTS
#define NSMALLPOSINTS           257
#endif
#ifndef NSMALLNEGINTS
#define NSMALLNEGINTS           5
#endif
#if NSMALLNEGINTS + NSMALLPOSINTS > 0
/* References to small integers are saved in this array so that they
   can be shared.
   The integers that are saved are those in the range
   -NSMALLNEGINTS (inclusive) to NSMALLPOSINTS (not inclusive).
*/
static PyIntObject *small_ints[NSMALLNEGINTS + NSMALLPOSINTS];
#endif
#ifdef COUNT_ALLOCS
Py_ssize_t quick_int_allocs;
Py_ssize_t quick_neg_int_allocs;
#endif

PyObject *
PyInt_FromLong(long ival)
{
    register PyIntObject *v;
#if NSMALLNEGINTS + NSMALLPOSINTS > 0
    if (-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS) {
        v = small_ints[ival + NSMALLNEGINTS];
        Py_INCREF(v);
#ifdef COUNT_ALLOCS
        if (ival >= 0)
            quick_int_allocs++;
        else
            quick_neg_int_allocs++;
#endif
        return (PyObject *) v;
    }
#endif
    if (free_list == NULL) {
        if ((free_list = fill_free_list()) == NULL)
            return NULL;
    }
    /* Inline PyObject_New */
    v = free_list;
    free_list = (PyIntObject *)Py_TYPE(v);
    PyObject_INIT(v, &PyInt_Type);
    v->ob_ival = ival;
    return (PyObject *) v;
}