How to load an old pickle file?

Question

I have an old pickle file, and when I try to load it, I get the error

ImportError: No module named oldname.submodule

This is because the oldname has been renamed to newname long ago.

How do I load this pickle file?

I can create a symbolic link from oldname to newname , but I wonder if there is a way to make modules oldname.* refer to newname.* without touching the filesystem?

Answer 1

Instead of a symlink which creates a second parallel module and an external dependency, add an entry in sys.modules . To test, I pickled a class instance then renamed the module

td@mintyfresh ~/tmp $ cat oldname.py
class Foo(object):
    def __init__(self):
        self.name = 'bar'

td@mintyfresh ~/tmp $ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import oldname
>>> foo = oldname.Foo()
>>> foo.name
'bar'
>>> import pickle
>>> pickle.dump(foo, open('test.pkl', 'wb'), 2)
>>> exit()
td@mintyfresh ~/tmp $ mv oldname.py newname.py
td@mintyfresh ~/tmp $ rm oldname.*

Now loading fails

td@mintyfresh ~/tmp $ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import newname
>>> import pickle
>>> foo = pickle.load(open('test.pkl', 'rb'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1090, in load_global
    klass = self.find_class(module, name)
  File "/usr/lib/python2.7/pickle.py", line 1124, in find_class
    __import__(module)
ImportError: No module named oldname

But if I duplicate the module in sys.modules, it works. Interestingly, the class instance has the new module name.

>>> import sys
>>> sys.modules['oldname'] = sys.modules['newname']
>>> foo = pickle.load(open('test.pkl', 'rb'))
>>> foo
<newname.Foo object at 0x7f6b837d3310>

Caveat emptor: If the class itself changed a lot pickle will fail anyway.