Can't pickle object error using SqlAlchemy

Question

I'm trying to store a mutable object in a postgresql database using SqlAlchemy. The class of the object is roughly:

class Data(Mutable, object):
    @classmethod
    def coerce(cls, key, value):
        return value

    def __init__(self, foo, bar):
        self._foo = foo     # foo is an array of short strings
        self._bar = bar     # bar is a single short string

    def append_to_foo(self, baz):
        self._foo.append(baz)
        self.changed()

    # Various other methods for mutating/accessing foo and bar, which call self.changed()
    # when they finish

The column definition is:

data = Column(Data.as_mutable(PickleType))

Whenever I try to add a row to the table including this column, I get the following error:

sqlalchemy.exc.StatementError: (builtins.AttributeError) Can't pickle local object 'WeakKeyDictionary.__init__.<locals>.remove' [SQL: "INSERT INTO table (id, user_id, data) VALUES (nextval('data_id_seq'), %(user_id)s, %(data)s) RETURNING data.id"] [parameters: [{'data': <mypkg.foo.Data object at 0x7f79b3b52c88>, 'user_id': 36}]]

I've made sure that the Data class itself is pickleable using the Python prompt; I can pickle.dumps and pickle.loads instances with no issues. Google produces no results, and I couldn't find any relevant bug reports.

I'm using SqlAlchemy 1.0.13 and Python 3.5.

Answer 1

From reading "Supporting Pickling" it would seem you must provide at least the __getstate__ method for custom mutable types:

The developer responsibility here is only to provide a __getstate__ method that excludes the _parents() collection from the pickle stream:

This is because the mutable extension places a weakref.WeakKeyDictionary upon the value object, which cannot be pickled. A minimal __getstate__ implementation is provided in the documentation:

class Data(Mutable, object):

    ...

    def __getstate__(self):
        d = self.__dict__.copy()
        d.pop('_parents', None)
        return d

Depending on how your type is implemented you may have to provide __setstate__ as well.