为什么我会收到持久性 sklearn 模型的 unpickling 错误？

Question

I have a trained decision tree model which I created in a google research notebook, and pickled like this:

pickle.dump(tree, open(tree_15.sav, 'wb'))

We have an online analysis project built from docker, creating the same environment that I used to create the model. (And some other packages for other functionality.) I want to import my model in that other project, with the following line:

predictive_model = pickle.load(open(self.sav_path, 'rb'), fix_imports=True, encoding='latin1')

But I get:

_pickle.UnpicklingError: invalid load key, '\xef'.

Funny thing is, I can not reproduce the same bug locally. I use a windows desktop (so I have problems with building from docker locally), but I use the same version of every package, the .sav file is not corrupted - hash checks out -, etc. It runs quite fine locally every time. The previous version of the same model ran online as well. I'm quite lost here to be honest.

Answer 1

Often, gzip 's open() doesn't have the issues that the built-in open() has, so why not try the following:

import gzip, pickle
with gzip.open(self.sav_path, 'rb') as f:
    predictive_model = pickle.load(f, fix_imports=True, encoding='latin1')