Use metaclass to allow forward declarations

Question

I want to do something decidedly unpythonic. I want to create a class that allows for forward declarations of its class attributes. (If you must know, I am trying to make some sweet syntax for parser combinators.)

This is the kind of thing I am trying to make:

a = 1
class MyClass(MyBaseClass):
    b = a  # Refers to something outside the class
    c = d + b  # Here's a forward declaration to 'd'
    d = 1  # Declaration resolved

My current direction is to make a metaclass so that when d is not found I catch the NameError exception and return an instance of some dummy class I'll call ForwardDeclaration . I take some inspiration from AutoEnum , which uses metaclass magic to declare enum values with bare identifiers and no assignment.

Below is what I have so far. The missing piece is: how do I continue normal name resolution and catch the NameError s:

class MetaDict(dict):
    def __init__(self):
        self._forward_declarations = dict()
    def __getitem__(self,  key):
        try:
            ### WHAT DO I PUT HERE ??? ###
            # How do I continue name resolution to see if the
            # name already exists is the scope of the class
        except NameError:
            if key in self._forward_declarations:
                return self._forward_declarations[key]
            else:
                new_forward_declaration = ForwardDeclaration()
                self._forward_declarations[key] = new_forward_declaration
                return new_forward_declaration

class MyMeta(type):
    def __prepare__(mcs, name, bases):
        return MetaDict()

class MyBaseClass(metaclass=MyMeta):
    pass

class ForwardDeclaration:
    # Minimal behavior
    def __init__(self, value=0):
        self.value = value
    def __add__(self, other):
        return ForwardDeclaration(self.value + other)

Answer 1

To start with:

    def __getitem__(self,  key):
        try:
            return super().__getitem__(key)
        except KeyError:
             ...

But that won't allow you to retrieve the global variables outside the class body. You can also use the __missin__ method which is reserved exactly for subclasses of dict:

class MetaDict(dict):
    def __init__(self):
        self._forward_declarations = dict()

    # Just leave __getitem__ as it is on "dict"
    def __missing__(self,  key):
        if key in self._forward_declarations:
            return self._forward_declarations[key]
        else:
            new_forward_declaration = ForwardDeclaration()
            self._forward_declarations[key] = new_forward_declaration
            return new_forward_declaration

As you can see, that is not that "UnPythonic" - advanced Python stuff such as SymPy and SQLAlchemy have to resort to this kind of behavior to do their nice magic - just be sure to get it very well documented and tested.

Now, to allow for global (module) variables, you have a to get a little out of the way - and possibly somthing that may not be avaliablein all Python implementations - that is: introspecting the frame where the class body is to get its globals:

import sys
...
class MetaDict(dict):
    def __init__(self):
        self._forward_declarations = dict()

    # Just leave __getitem__ as it is on "dict"
    def __missing__(self,  key):
        class_body_globals = sys._getframe().f_back.f_globals
        if key in class_body_globals:
             return class_body_globals[key]
        if key in self._forward_declarations:
            return self._forward_declarations[key]
        else:
            new_forward_declaration = ForwardDeclaration()
            self._forward_declarations[key] = new_forward_declaration
            return new_forward_declaration

Now that you are here - your special dictionaries are good enough to avoid NameErrors, but your ForwardDeclaration objects are far from smart enough - when running:

a = 1
class MyClass(MyBaseClass):
    b = a  # Refers to something outside the class
    c = d + b  # Here's a forward declaration to 'd'
    d = 1 

What happens is that c becomes a ForwardDeclaration object, but summed to the instant value of d which is zero. On the next line, d is simply overwritten with the value 1 and is no longer a lazy object. So you might just as well declare c = 0 + b .

To overcome this, ForwardDeclaration has to be a class designed in a smartway, so that its values are always lazily evaluated, and it behaves as in the "reactive programing" approach: ie: updates to a value will cascade updates into all other values that depend on it. I think giving you a full implementation of a working "reactive" aware FOrwardDeclaration class falls off the scope of this question. - I have some toy code to do that on github at https://github.com/jsbueno/python-react , though.

Even with a proper "Reactive" ForwardDeclaration class, you have to fix your dictionary again so that the d = 1 class works:

class MetaDict(dict):
    def __init__(self):
        self._forward_declarations = dict()

    def __setitem__(self, key, value):
        if key in self._forward_declarations:
            self._forward_declations[key] = value
            # Trigger your reactive update here if your approach is not
            # automatic
            return None
         return super().__setitem__(key, value)
    def __missing__(self,  key):
        # as above

And finally, there is a way to avoid havign to implement a fully reactive aware class - you can resolve all pending FOrwardDependencies on the __new__ method of the metaclass - (so that your ForwardDeclaration objects are manually "frozen" at class creation time, and no further worries - )

Something along:

from functools import reduce

sentinel = object()
class ForwardDeclaration:
    # Minimal behavior
    def __init__(self, value=sentinel, dependencies=None):
        self.dependencies = dependencies or []
        self.value = value
    def __add__(self, other):
        if isinstance(other, ForwardDeclaration):
             return ForwardDeclaration(dependencies=self.dependencies + [self])
        return ForwardDeclaration(self.value + other)

class MyMeta(type):
    def __new__(metacls, name, bases, attrs):
         for key, value in list(attrs.items()):
              if not isinstance(value, ForwardDeclaration): continue
              if any(v.value is sentinel for v in value.dependencies): continue
              attrs[key] = reduce(lambda a, b: a + b.value, value.dependencies, 0) 

         return super().__new__(metacls, name, bases, attrs)
    def __prepare__(mcs, name, bases):
        return MetaDict()

And, depending on your class hierarchy and what exactly you are doing, rememebr to also update one class' dict _forward_dependencies with the _forward_dependencies created on its ancestors. AND if you need any operator other than + , as you will have noted, you will have to keep information on the operator itself - at this point, hou might as well jsut use sympy .