why defined '__new__' and '__init__' all in a class

Question

i think you can defined either ' __init__ ' or ' __new__ ' in a class,but why all defined in django.utils.datastructures.py.

my code:

class a(object):
    def __init__(self):
        print  'aaa'
    def __new__(self):
        print 'sss'

a()#print 'sss'

class b:
    def __init__(self):
        print  'aaa'
    def __new__(self):
        print 'sss'
b()#print 'aaa'

datastructures.py:

class SortedDict(dict):
    """
    A dictionary that keeps its keys in the order in which they're inserted.
    """
    def __new__(cls, *args, **kwargs):
        instance = super(SortedDict, cls).__new__(cls, *args, **kwargs)
        instance.keyOrder = []
        return instance

    def __init__(self, data=None):
        if data is None:
            data = {}
        super(SortedDict, self).__init__(data)
        if isinstance(data, dict):
            self.keyOrder = data.keys()
        else:
            self.keyOrder = []
            for key, value in data:
                if key not in self.keyOrder:
                    self.keyOrder.append(key)

and what circumstances the SortedDict.__init__ will be call.

thanks

Answer 1

You can define either or both of __new__ and __init__ .

__new__ must return an object -- which can be a new one (typically that task is delegated to type.__new__ ), an existing one (to implement singletons, "recycle" instances from a pool, and so on), or even one that's not an instance of the class. If __new__ returns an instance of the class (new or existing), __init__ then gets called on it; if __new__ returns an object that's not an instance of the class, then __init__ is not called.

__init__ is passed a class instance as its first item (in the same state __new__ returned it, ie, typically "empty") and must alter it as needed to make it ready for use (most often by adding attributes).

In general it's best to use __init__ for all it can do -- and __new__ , if something is left that __init__ can't do, for that "extra something".

So you'll typically define both if there's something useful you can do in __init__ , but not everything you want to happen when the class gets instantiated.

For example, consider a class that subclasses int but also has a foo slot -- and you want it to be instantiated with an initializer for the int and one for the .foo . As int is immutable, that part has to happen in __new__ , so pedantically one could code:

>>> class x(int):
...   def __new__(cls, i, foo):
...     self = int.__new__(cls, i)
...     return self
...   def __init__(self, i, foo):
...     self.foo = foo
...   __slots__ = 'foo',
... 
>>> a = x(23, 'bah')
>>> print a
23
>>> print a.foo
bah
>>> 

In practice, for a case this simple, nobody would mind if you lost the __init__ and just moved the self.foo = foo to __new__ . But if initialization is rich and complex enough to be best placed in __init__ , this idea is worth keeping in mind.

Answer 2

__new__ and __init__ do completely different things. The method __init__ initiates a new instance of a class --- it is a constructor. __new__ is a far more subtle thing --- it can change arguments and, in fact, the class of the initiated object. For example, the following code:

class Meters(object):
    def __new__(cls, value):
        return int(value / 3.28083)

If you call Meters(6) you will not actually create an instance of Meters , but an instance of int . You might wonder why this is useful; it is actually crucial to metaclasses, an admittedly obscure (but powerful) feature.

You'll note that in Python 2.x, only classes inheriting from object can take advantage of __new__ , as you code above shows.

The use of __new__ you showed in django seems to be an attempt to keep a sane method resolution order on SortedDict objects. I will admit, though, that it is often hard to tell why __new__ is necessary. Standard Python style suggests that it not be used unless necessary (as always, better class design is the tool you turn to first).

Answer 3

My only guess is that in this case, they (author of this class) want the keyOrder list to exist on the class even before SortedDict.__init__ is called.

Note that SortedDict calls super() in its __init__ , this would ordinarily go to dict.__init__ , which would probably call __setitem__ and the like to start adding items. SortedDict.__setitem__ expects the .keyOrder property to exist, and therein lies the problem (since .keyOrder isn't normally created until after the call to super() .) It's possible this is just an issue with subclassing dict because my normal gut instinct would be to just initialize .keyOrder before the call to super() .

The code in __new__ might also be used to allow SortedDict to be subclassed in a diamond inheritance structure where it is possible SortedDict.__init__ is not called before the first __setitem__ and the like are called. Django has to contend with various issues in supporting a wide range of python versions from 2.3 up; it's possible this code is completely un-neccesary in some versions and needed in others.

---

There is a common use for defining both __new__ and __init__ : accessing class properties which may be eclipsed by their instance versions without having to do type(self) or self.__class__ (which, in the existence of metaclasses, may not even be the right thing).

For example:

class MyClass(object):
    creation_counter = 0

    def __new__(cls, *args, **kwargs):
        cls.creation_counter += 1
        return super(MyClass, cls).__new__(cls)

    def __init__(self):
         print "I am the %dth myclass to be created!" % self.creation_counter

Finally, __new__ can actually return an instance of a wrapper or a completely different class from what you thought you were instantiating. This is used to provide metaclass-like features without actually needing a metaclass.

Answer 4

In my opinion, there was no need of overriding __new__ in the example you described. Creation of an instance and actual memory allocation happens in __new__ , __init__ is called after __new__ and is meant for initialization of instance serving the job of constructor in classical OOP terms. So, if all you want to do is initialize variables, then you should go for overriding __init__ . The real role of __new__ comes into place when you are using Metaclasses. There if you want to do something like changing attributes or adding attributes, that must happen before the creation of class, you should go for overriding __new__ .

Consider, a completely hypothetical case where you want to make some attributes of class private, even though they are not defined so (I'm not saying one should ever do that).

class PrivateMetaClass(type):
      def __new__(metaclass, classname, bases, attrs):
          private_attributes = ['name', 'age']

          for private_attribute in private_attributes:
              if attrs.get(private_attribute):
                 attrs['_' + private_attribute] = attrs[private_attribute]
                 attrs.pop(private_attribute)

          return super(PrivateMetaClass, metaclass).__new__(metaclass, classname, bases, attrs)


class Person(object):

      __metaclass__ = PrivateMetaClass

      name = 'Someone'
      age = 19

person = Person()
>>> hasattr(person, 'name')
False
>>> person._name
'Someone'

Again, It's just for instructional purposes I'm not suggesting one should do anything like this.