Python mixin/decorator/__metaclass__ for base class enhancement

Question

I am implementing a content-aware caching system for a Django REST API. I would like to develop a component which can be added to existing views that would modify the behavior of the base class by checking the cache and falling back to the base class behavior on a miss.

basically, I have something like this:

class Base:
   def get(self, request, *args, **kwargs):
       ....
       return Response

class AnotherBase:
   def get(self, request, *args, **kwargs):
       .... 
       return Response

class Derived(Base):
    pass

class OtherDerived(AnotherBase):
    pass

and my initial thought was to do something along the lines of

class Cacheable:
    def get(self, request, *args, **kwargs):
       cache_key = self.get_cache_key(request)
       base_get = #.... and this is the problem
       return cache.get(cache_key, base_get(request, *args, **kwargs))

    def get_cache_key(self, request):
       # .... do stuff

class Derived(Cacheable, Base):
    pass

class AnotherDerived(Cacheable, AnotherBase):
    pass

So clearly this doesn't work, as I don't know how, or if it's possible, or if it's advisable to access the sibling superclass(es) from a mixin.

My goal is an implementation that allows me to add caching behavior to existing views without touching the internals of the existing classes. Given a view class, C , st C.get(request, *args, **kwargs) -> Response , is there a function, F , st F(C).get(... does the cache check before falling back to C.get ? And in this quasi-formal notation, we'll say that adding a mixin to the leftmost parent class in the class definition counts as a function.

Is it more appropriate to use method decorators? or how would a class decorator work?

And then I've seen references to __metaclass__ in researching this, but I'm not clear on what that approach looks like.

This is Python 3.6

Answer 1

Simple example:

def Base:

    def _get_data(self):
        # get the data eg from database
        return self._get_data_native()

    def get(self, request, *args, **kwargs):
        return Response(self._get_data())

def Cacheable(Base):

    def _get_data(self):
        # get from cache ...
        result = ...
        if result is None:
            # or from base ...
            result = ...

        return result

def Derived(Cacheable):

    def _get_data_native(self):
        # get the data eg from database
        ...

By inheriting from Cacheable, you include the caching here, because _get_data is overwritten there.

For this problem, you don't need metaclasses or decorators, if you want to just add caching at one place.

Of course, a decorator could be used for including caching in an even more generic way.

See for example this answer: Is there a decorator to simply cache function return values?

Answer 2

The answer was a decorator and some Django -specific libraries.

from django.utils.decorators import method_decorator
from django.core.cache import cache

def cached_get(cache_key_func=None):
    """
    Decorator to be applied via django.utils.decorators.method_decorator
    Implements content-aware cache fetching by decorating the "get" method
    on a django View
    :param cache_key_func: a function of fn(request, *args, **kwargs) --> String
    which determines the cache key for the request
    """
    def decorator(func):
        def cached_func(request, *args, **kwargs):
            assert cache_key_func is not None, "cache_key_function is required"
            key = cache_key_func(request, *args, **kwargs)
            result = cache.get(key)
            if result is None:
                return func(request, *args, **kwargs)
            return Response(result)
        return cached_func
    return decorator

@method_decorator(cached_get(cache_key_func=get_cache_key), name="get")
class SomeView(BaseView):
    ...

def get_cache_key(request):
    # do arbitrary processing on request, the following is the naïve melody
    key =  urllib.urlencode(request.query_params)
    return key 

So the solution is to use Django's built-in method_decorator which applies its first argument, a decorator, to the decorated class's method, named by the second argument, name , to method_decorator . I define a higher-order function, cached_get , which takes another function as its argument, and returns a curried function (closure, so called). By calling this, with the function get_cache_key (and not, mind you, invoking that function) I have a decorator that will be applied to the 'get' method on SomeView .

The decorator itself is a straightforward Python decorator -- in this application, it is cached_func and the original, undecorated get method is func . Thus, cached_func replaces SomeView.get , so when SomeView.get is called, it first checks the cache, but falls back to the undecorated method on a miss.

I'm hopeful this approach provides a balance of generic applicability with content-aware key derivation.

Answer 3

My two cents:

1. You're walking into obscure territory here. Get familiar with all the related concepts, try a few, then decide.
2. Here is a good tutorial about metaclasses.
3. Here there's one about decorators.
4. I'm in no way affiliated to that site.