类装饰器问题，或者：Python如何区分静态方法和方法？

Question

(I need a Python 3 internals guru, please)

I have a class decorator that modifies some functions but not others. Simplified example:

import functools
import inspect
import types

def mydecorator(myobj):
    @functools.wraps(myobj)
    def decorated_method(*args, **kwargs):
        print("I'm decorated!")
        return myobj(*args, **kwargs)

    if inspect.isclass(myobj):  # act as class decorator
        for name, obj in myobj.__dict__.items():
            if name == "add":
                setattr(myobj, name, types.MethodType(mydecorator(obj), myobj))
        return myobj  # return decorated class
    elif inspect.isfunction(myobj):  # act as function decorator
        return decorated_method
    else:
        assert False, "can decorate only classes and functions"

So this will modify any add method to print "I'm decorated" before it runs.

We'll apply it to this class:

class MyClass:
    def add(self, x, y): return x + y
    def mul(self, x, y): return x * y

and it works alright. We do

#--- try out undecorated MyClass:
print("MyClass.add:", MyClass.add, "MyClass.mul:", MyClass.mul)
print("3+4 =", MyClass().add(3, 4), "3*4 =", MyClass().mul(3, 4), )

#--- decorate MyClass:
print("MyClass = mydecorator(MyClass)")
MyClass = mydecorator(MyClass)

#--- try out decorated MyClass in the same manner:
print("MyClass.add:", MyClass.add, "MyClass.mul:", MyClass.mul)
print("3+4 =", MyClass().add(3, 4), "3*4 =", MyClass().mul(3, 4), )

and get this output (from CPython 3.6.7 on Linux)

MyClass.add: <function MyClass.add at 0x7faededda0d0> MyClass.mul: <function MyClass.mul at 0x7faededda158>
3+4 = 7 3*4 = 12
MyClass = mydecorator(MyClass)
MyClass.add: <bound method MyClass.add of <class '__main__.MyClass'>>  MyClass.mul: <function MyClass.mul at 0x7faededda158>
I'm decorated!
3+4 = 7 3*4 = 12

So mul stays a plain function while the decorated add turns into a bound method. The decoration works correctly.

But when I now change the method such that add calls mul (ignore the fact this does not make much sense) as follows:

class MyClass:
    def add(self, x, y): z = self.mul(x, y); return x + y
    def mul(self, x, y): return x * y

the output turns into this:

MyClass.add: <function MyClass.add at 0x7fbc760870d0> MyClass.mul: <function MyClass.mul at 0x7fbc76087158>
3+4 = 7 3*4 = 12
MyClass = mydecorator(MyClass)
MyClass.add: <bound method MyClass.add of <class '__main__.MyClass'>> MyClass.mul: <function MyClass.mul at 0x7fbc76087158>
I'm decorated!
Traceback (most recent call last):
  File "tryout.py", line 34, in <module>
    print("3+4 =", MyClass().add(3, 4), "3*4 =", MyClass().mul(3, 4), )
  File "tryout.py", line 16, in decorated_method
    return myobj(*args, **kwargs)
  File "tryout.py", line 7, in add
    def add(self, x, y): z = self.mul(x, y); return x + y  # round 2
TypeError: mul() missing 1 required positional argument: 'y'

It turns out that mul (although it is the same as before!) is now being called as if it were a @staticmethod : self is not passed.

I have plenty of questions:

1. Where does this astonishing effect come from?
2. What object is add bound to?
3. How does Python internally discriminate a normal method from a @classmethod or a @staticmethod ?
4. What does types.MethodType really mean?
5. What would I write in its place to get a normal method, class method, or static method, respectively?
6. Where would I have found the documentation of all this?
7. Which of the answers pertain to Python properties as opposed to CPython implementation details?

Answer 1

The problem is that you should not replace the function add with a bound method. The way methods work is that a function object has a __get__ method which, in the case of an instance method, returns a bound method for you to be called on the provided arguments. That is, given

class MyClass:
    def add(self, x, y): 
        return x + y
    def mul(self, x, y):
        return x * y

o = MyClass()

a call like o.add(3,5) is equivalent to type(o).__dict__['add'].__get__(o, type(o))(3,5) .

Your decorator should also simply return a new function, rather than a method object, and let its __get__ method do its job.

Your new decorator, with some simplifications:

def mydecorator(myobj):
    @functools.wraps(myobj)
    def decorated_method(*args, **kwargs):
        print("I'm decorated!")
        return myobj(*args, **kwargs)

    # Decorating a function
    if inspect.isfunction(myobj):
        return decorated_method

    # Decorating a class
    if inspect.isclass(myobj):
        if "add" in myobj.__dict__:
            setattr(myobj, "add", mydecorator(obj))
            # Or just setattr(myobj, "add", decorated_method),
            # unless you think myobj.add might be a nested class
        return myobj

    # Anything else is type error.
    raise TypeError("can decorate only classes and functions")

---

Addressing some of your other questions...

How does Python internally discriminate a normal method from a @classmethod or a @staticmethod?

The classmethod and staticmethod objects return objects that have different __get__ methods than a regular function object.

Where would I have found the documentation of all this?

The Descriptor How-to Guide is a good place to start. It describes the descriptor protocol, as well as examples of how things like properties and methods make use of it.

Answer 2

To add to @chepner's great answer and to answer your point 4.

What does types.MethodType really mean?

This special type allows us to add methods to already created instances, in your case there's no instance, hence when you pass myobj to it, you're basically setting __self__ of the wrapper to its class rather than using its instance.

Let's take the simpler version of your class, without the decorator:

class MyClass:
    def add(self, x, y):
        z = self.mul(x, y);
        return x + y
    def mul(self, x, y): return x * y

Now:

>>> ins = MyClass()
>>> ins.add.__func__
<function MyClass.add at 0x7f483050cf28>
>>> ins.add.__self__
<__main__.MyClass object at 0x7f48304ef390>

As you can see, the add is a whole new object that has now info on which function to call and what to pass as first argument.

>>> ins.add(1, 2)
3
>>> ins.add.__func__(ins.add.__self__, 1, 2)
3

Now when we do, what you did:

>>> MyClass.add = types.MethodType(ins.add.__func__, MyClass)

This now passes the class to the function add instead of the instance:

>>> ins = MyClass()
>>> ins.add.__self__
<class '__main__.MyClass'>

which means the self inside add now is not the actual instance, but the class. Which means self.mul call there is equivalent to:

>>> MyClass.mul(1, 2)
...
TypeError: mul() missing 1 required positional argument: 'y'