traceback behaviour for __init__ errors when using __call__ in metaclasses?

Question

Using the following code:

class Meta(type):
    def __new__(mcl, name, bases, nmspc):
        return super(Meta, mcl).__new__(mcl, name, bases, nmspc)

class TestClass(object):
    __metaclass__ = Meta

    def __init__(self):
        pass

t = TestClass(2) # deliberate error

produces the following:

Traceback (most recent call last):
  File "foo.py", line 12, in <module>
    t = TestClass(2)
TypeError: __init__() takes exactly 1 argument (2 given)

However using __call__ instead of __new__ in the following code:

class Meta(type):
    def __call__(cls, *args, **kwargs):
        instance =  super(Meta, cls).__call__(*args, **kwargs)
        # do something
        return instance

class TestClass(object):
    __metaclass__ = Meta

    def __init__(self):
        pass

t = TestClass(2) # deliberate error

gives me the following traceback:

Traceback (most recent call last):
  File "foo.py", line 14, in <module>
    t = TestClass(2)
  File "foo.py", line 4, in __call__
    instance =  super(Meta, cls).__call__(*args, **kwargs)
TypeError: __init__() takes exactly 1 argument (2 given)

1. Does type also trigger the __init__ of the class from its __call__ or is the behaviour changed when I add the metaclass?
2. Both __new__ and __call__ are being run by the call to the class constructor. Why is __call__ showing up in the error message but not __new__ ?
3. Is there a way of suppressing the lines of the traceback showing the __call__ for the metaclass here? ie when the error is in the call to the constructor and not the __call__ code?

Answer 1

Lets see if I can answer your three questions:

Does type also trigger the __init__ of the class from its __call__ or is the behaviour changed when I add the metaclass?

The default behavior of type.__call__ is to create a new object with cls.__new__ (which may be inherited from object.__new__ , or call it with super ). If the object returned from cls.__new__ is an instance of cls , type.__call__ will then run cls.__init__ on it.

If you define your own __call__ method in a custom metaclass, it can do almost anything. Usually though you'll call type.__call__ at some point (via super ) and so the same behavior will happen. This isn't required though. You can return anything from a metaclass's __call__ method.

Both __new__ and __call__ are being run by the call to the class constructor. Why is __call__ showing up in the error message but not __new__ ?

You're misunderstanding what Meta.__new__ is for. The __new__ method in a metaclass is not called when you make an instance of the normal class. It is called when you make an instance of the metaclass, which is the class object itself.

Try running this code, to better understand what is going on:

print("Before Meta")

class Meta(type):
    def __new__(meta, name, bases, dct):
        print("In Meta.__new__")
        return super(Meta, meta).__new__(meta, name, bases, dct)

    def __call__(cls, *args, **kwargs):
        print("In Meta.__call__")
        return super(Meta, cls).__call__(*args, **kwargs)

print("After Meta, before Cls")

class Cls(object):
    __metaclass__ = Meta

    def __init__(self):
        print("In Cls.__init__")

print("After Cls, before obj")

obj = Cls()

print("Bottom of file")

The output you'll get is:

Before Meta
After Meta, before Cls
In Meta.__new__
After Cls, before obj
In Meta.__call__
In Cls.__init__
Bottom of file

Note that Meta.__new__ is called where the regular class Cls is defined, not when the instance of Cls is created. The Cls class object is in fact an instance of Meta , so this makes some sense.

The difference in your exception tracebacks comes from this fact. When the exception occurs, the metaclass's __new__ method has long since finished (since if it didn't, there wouldn't have been a regular class to call at all).

Is there a way of suppressing the lines of the traceback showing the __call__ for the metaclass here? ie when the error is in the call to the constructor and not the __call__ code?

Yes and no. It's probably possible, but its almost certainly a bad idea. Python's stacktraces will, by default, show you the full call stack (excluding builtin stuff that's implemented in C, rather than Python). That's their purpose. The problem causing an exception in your code is not always going to be in the last call, even in less confusing areas than metaclasses.

Consider this trivial example:

def a(*args):
    b(args) # note, no * here, so a single tuple will be passed on

def b(*args):
    c(*args):

def c():
    print(x)

a()

In this code, there's an error in the a function, but an exception is only raised when b calls c with the wrong number of arguments.

I suppose if you needed to you could pretty things up a bit by editing the data in the stack trace object somewhere, but if you do that automatically it is likely to make things much more confusing if you ever encounter an actual error in the metaclass code.

Answer 2

In fact, what the interpreter is complaining about is that you are not passing arg to __init__ .

You should do:

t = TestClass('arg')

or:

class TestClass(object):
    __metaclass__ = Meta

    def __init__(self):
        pass

t = TestClass()