How to automate the delegation of __special_methods__ in Python?

Question

Let spam be an instance of some class Spam , and suppose that spam.ham is an object of some built-in type, say dict . Even though Spam is not a subclass of dict , I would like its instances to have the same API as a regular dict (ie the same methods with the same signatures), but I want to avoid typing out a bazillion boilerplate methods of the form:

    def apimethod(self, this, that):
        return self.ham.apimethod(this, that)

I tried the following:

class Spam(object):
    def __init__(self):
        self.ham = dict()

    def __getattr__(self, attr):
        return getattr(self.ham, attr)

...but it works for "regular" methods, like keys and items , but not for special methods , like __setitem__ , __getitem__ , and __len__ :

>>> spam = Spam()
>>> spam.keys()
[]
>>> spam['eggs'] = 42
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Spam' object does not support item assignment
>>> spam.ham['eggs'] = 42
>>> foo.items()
[('eggs', 42)]
>>> spam['eggs']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Spam' object is not subscritable
>>> len(spam)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Spam' object has no len()

All the special methods I tried produced similar errors.

How can I automate the definition of special methods (so that they get referred to the delegate)?

Clarification: I'm not necessarily looking for solutions that leverage the standard method lookup sequence. My goal here is to minimize boilerplate code.

Thanks!

Answer 1

This may not be helpful if you need a solution that prohibits metaclasses as well, but here is the solution I came up with:

def _wrapper(func):
    def _wrapped(self, *args, **kwargs):
        return getattr(self.ham, func)(*args, **kwargs)
    return _wrapped

class DictMeta(type):
    def __new__(cls, name, bases, dct):
        default_attrs = dir(object)
        for attr in dir(dict):
            if attr not in default_attrs:
                dct[attr] = _wrapper(attr)
        return type.__new__(cls, name, bases, dct)

class Spam(object):
    __metaclass__ = DictMeta
    def __init__(self):
        self.ham = dict()

Seems to do what you're looking for:

>>> spam = Spam()
>>> spam['eggs'] = 42
>>> spam.items()
[('eggs', 42)]
>>> len(spam)
1
>>> spam.ham
{'eggs': 42}

If on Python 3.x use class Spam(object, metaclass=DictMeta) and remove the __metaclass__ line from the body of Spam .

Answer 2

This looks like a job for ... a metaclass!

def make_method(p, m):
    def method(self, *a, **k):
        return getattr(getattr(self, p),m)(*a, **k)
    return method

class Proxier(type):
    def __new__(cls, name, bases, dict):
        objs = dict.get('proxyobjs', [])
        if objs:
            old_init = dict.get('__init__', lambda self: None)
            def new_init(self, *a, **k):
                for (n,v) in objs.iteritems():
                    setattr(self, n, v())
                old_init(self, *a, **k)
            dict['__init__'] = new_init
            meths = dict.get('proxymethods', {})
            for (proxyname, methnames) in meths.iteritems():
                for methname in methnames:                
                    dict[methname] = make_method(proxyname, methname)
        return super(Proxier, cls).__new__(cls, name, bases, dict)


class Spam(object):
    __metaclass__ = Proxier
    proxyobjs = {'ham': dict,
                 'eggs': list,
                 }
    proxymethods = {'ham': ('__setitem__', '__getitem__', '__delitem__'),
                    'eggs': ('__contains__', 'append')
                    } 

It works!

In [28]: s = Spam()

In [29]: s[4] = 'hi'

In [30]: s.append(3)

In [31]: 3 in s
Out[31]: True

In [32]: 4 in s
Out[32]: False

In [33]: s[4]
Out[33]: 'hi'

Note that you have to specify what parts of the interface you're using (otherwise, why not just inherit?). So we have __contains__ from list , and __getitem__ from dict , and the __iter__ from neither. (And only one way to mutate the underlying list, using append but not extend or __delitem__ .) So (like Martian) I'm not sure how useful this will be.

Answer 3

Attribute access for special methods doesn't obey normal attribute access rules, basically those methods MUST exist at class level, read http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes

So you need to add all those methods either manually or you can add them to class programmatically and best way to do that is thru metaclass. Also note that I am not adding all methods in dict but only special methods because rest can be easily redirected thru __getattr__

def redirect(methodname):
    def _redirect(self, *args, **kwargs):
        print "redirecting",methodname
        method = getattr(self.ham, methodname) 
        return method(*args, **kwargs)

    return _redirect

class DictRedirect(object):
    def __new__(cls, name, bases, attrs):

        # re-create all special methods from dict
        dict_attr_names = set(dir(dict))
        common_names = set(dir(cls))
        for methodname in dict_attr_names-common_names:
            if not methodname.startswith('__'):
                continue
            attrs[methodname] = redirect(methodname)
        return type(name, bases, attrs)

class Spam(object):
    __metaclass__ = DictRedirect

    def __init__(self):
        self.ham = dict()

    def __getattr__(self, name):
        return getattr(self.ham, name)

spam = Spam()
spam['eggs'] = 'yolk'
print 'keys =',spam.keys()
print spam['eggs']

output:

redirecting __setitem__
keys = ['eggs']
redirecting __getitem__
yolk

Disclaimer: IMO this is too much magic and should be avoided except for having fun :)

Answer 4

Not sure __getattribute__ will help, but the reason is the special methods are looked up in the class not in the instance: http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes , as for example the special methods like __getattr__ and __getattribute__ themselves have to be looked up somewhere.

Proxying like this seems asking for trouble to me without careful thinking, for example how should things like __dict__ and __class__ behave and about possible method conflicts if your wrapper happens to have any methods, and sure there are other problems.

Re: is-a vs. has-a:

If you just duplicate whole interface of contained member, it seems like anti-pattern to me, as that's what inheritance is for. What if you have a two has-a relations to two dict objects?

In has-a relation, one usually picks useful methods often exporting them under different names to make sensible API. So instead Spam.append(item) you would have Spam.addBot(bot) .