Python caching attributes in object with __slots__

Question

I am trying to cache a computationally expensive property in a class defined with the __slots__ attribute.

Any idea, how to store the cache for later use? Of course the usual way to store a dictionary in instance._cache would not work without __dict__ being defined. For several reasons i do not want to add a '_cache' string to __slots__ .

I was thinking whether this is one of the rare use cases for global. Any thoughts or examples on this matter?

Answer 1

You don't quite need a global; you can store the cache as a class property and still define the expensive property as a property .

class Foo(object):
    __slots__ = ('a', 'b', 'c')
    expensive_cache = {}

    @property
    def expensive(self):
        if self not in self.expensive_cache:
            self.expensive_cache[self] = self._compute_expensive()
        return self.expensive_cache[self]

    def _compute_expensive(self):
        print("Computing expensive property for {}".format(self))
        return 3

f = Foo()
g = Foo()
print(f.expensive)
print("===")
print(f.expensive)
print("===")
print(g.expensive)

If you run this code, you can see that _compute_expensive is run only once, the first time you access expensive for each distinct object.

$ python3 tmp.py
Computing expensive property for <__main__.Foo object at 0x102861188>
3
===
3
===
Computing expensive property for <__main__.Foo object at 0x1028611c8>
3

Answer 2

There is no magic possible there - ou want to store a value, so you need a place to store your value. You can't just decide "I won't have an extra entry on my __slots__ because it is not elegant" - you don't need to call it _cached : give it whatever name you want, but these cached values are something you want to exist in each of the object's instances, and therefore you need an attribute.

You can cache in a global (module level) dictionary, in which the keys are id(self) - but that would be a major headache to keep synchronized when instances are deleted. (The same thing is true for a class-level dictionary, with the further downside of it still be visible on the instance).

TL;DR : the "one and obvious way to do it" is to have a shadow attribute, starting with "_" to keep the values you want cached, and declare these in __slots__ . (If you use a _cached dictionary per instance, you loose the main advantage from __slots__ , that is exactly not needing one dictionary per instance).

Answer 3

Something like Borg pattern can help. You can alterate the status of your instance in the __init__ or __new__ methods.