How to pickle an instance of a class which is written inside a function?

Question

Suppose I have this snippet inside a module

def func(params):
   class MyClass(object):
       pass

How can I pickle an instance of the class MyClass ?

Answer 1

You can't, because picklable object's class definitions must reside in an imported module's scope. Just put your class inside module scope and you are good to go.

That said, in Python there is very little that can't be achieved with a bit of hacking the insides of the machinery (sys.modules in this case), but I wouldn't recommend that.

Answer 2

The MyClass definition is local variable for the func function. You cannot directly create an instance of it, but you can map it's functions to a new class, and then to use the new class as it is the original one. Here's an example:

def func(params):
    class MyClass(object):
        some_param = 100
        def __init__(self, *args):
            print "args:", args
        def blabla(self):
            self.x = 123
            print self.some_param
        def getme(self):
            print self.x

func.func_code is the code of the func function, and func.func_code.co_consts[2] contains the bytecode of the MyClass definition:

In : func.func_code.co_consts
Out: 
(None,
 'MyClass',
 <code object MyClass at 0x164dcb0, file "<ipython-input-35-f53bebe124be>", line 2>)

So we need the bytecode for the MyClass functions:

In : eval(func.func_code.co_consts[2])
Out: 
{'blabla': <function blabla at 0x24689b0>,
 '__module__': '__main__',
 'getme': <function getme at 0x2468938>,
 'some_param': 100,
 '__init__': <function __init__ at 0x219e398>}

And finally we create a new class with metaclass, that assigns the MyClass functions to the new class:

def map_functions(name, bases, dict):
    dict.update(eval(func.func_code.co_consts[2]))
    return type(name, bases, dict)

class NewMyClass(object):
    __metaclass__ = map_functions

n = NewMyClass(1, 2, 3, 4, 5)
>> args: (1, 2, 3, 4, 5)

n.blabla()
>> 100

n.getme()
>> 123

Answer 3

You can work around the pickle requirement that class definitions be importable by including the class definition as a string in the data pickled for the instance and exec() uting it yourself when unpickling by adding a __reduce__() method that passes the class definition to a callable. Here's a trivial example illustrating what I mean:

from textwrap import dedent

# Scaffolding
definition = dedent('''
    class MyClass(object):
        def __init__(self, attribute):
            self.attribute = attribute
        def __repr__(self):
            return '{}({!r})'.format(self.__class__.__name__, self.attribute)
        def __reduce__(self):
            return instantiator, (definition, self.attribute)
''')

def instantiator(class_def, init_arg):
    """ Create class and return an instance of it. """
    exec(class_def)
    TheClass = locals()['MyClass']
    return TheClass(init_arg)

# Sample usage
import pickle
from io import BytesIO

stream = BytesIO()  # use a memory-backed file for testing

obj = instantiator(definition, 'Foo')  # create instance of class from definition
print('obj: {}'.format(obj))
pickle.dump(obj, stream)

stream.seek(0) # rewind

obj2 = pickle.load(stream)
print('obj2: {}'.format(obj2))

Output:

obj: MyClass('Foo')
obj2: MyClass('Foo')

Obviously it's inefficient to include the class definition string with every class instance pickled, so that redundancy may make it impractical, depending on the the number of class instances involved.

Answer 4

This is somewhat tough to do because the way Pickle does with objects from user defined classes by default is to create a new instance of the class - using the object's __class__.__name__ attribute to retrieve its type in the object's original module. Which means: pickling and unpickling only works (by default) for classes that have well defined names in the module they are defined.

When one defines a class inside a function, usulay there won't be a module level (ie global) variable holding the name of each class that was created inside the function.

The behavior for pickle and npickle can be customized through the __getstate__ and __setstate__ methods on the class - check the docs - but even them, doing it right for dynamic class can be tricky , but I managed to create a working implementation of it for another SO question - -check my answer here: Pickle a dynamically parameterized sub-class