使用 joblib 加载腌制 scikit-learn 模型时出现 KeyError

Question

I have an object that contains within it two scikit-learn models, an IsolationForest and a RandomForestClassifier , that I would like to pickle and later unpickle and use to produce predictions. Apart from the two models, the object contains a couple of StandardScaler s and a couple of Python lists.

Pickling this object using joblib is unproblematic, but when I try to unpickle it later I get the following exception:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/(...)/python3.5/site-packages/joblib/numpy_pickle.py", line 578, in load
   obj = _unpickle(fobj, filename, mmap_mode)
 File "/home/(...)/python3.5/site-packages/joblib/numpy_pickle.py", line 508, in _unpickle
   obj = unpickler.load()
 File "/usr/lib/python3.5/pickle.py", line 1039, in load
   dispatch[key[0]](self)
KeyError: 0

The same application both pickles and unpickles the object, so the versions of scikit-learn , joblib and other libraries are the same. I'm not sure where to start debugging, given the vague error. Any ideas or pointers?

Answer 1

The solution to this was pretty banal: Without being aware of it I was using the version of joblib in sklearn.externals.joblib for the pickling, but a newer version of joblib for unpickling the object. The problem was resolved when I used the newer version of joblib for both tasks.

Answer 2

和我一起，碰巧我使用from sklearn.externals import joblib导出模型并尝试使用import joblib加载。

Answer 3

Mine was interesting. I was working with git-lfs and thus the files had been changed and joblib couldn't open them. So I needed to run git lfs pull to get actual files. So if you are using compatible joblib versions, make sure your files are not changed somehow!

Answer 4

对我来说，使用相同版本的 joblib 进行转储和加载，但我将文件保存在 python 3.7.4 下并尝试使用引发相同 KeyError 的 python 3.7.6 加载。

Answer 5

In my case, I was trying to load an XGB. I found outXGB is not compatible with other sklearn models, so I did the following:

from xgboost import *
import joblib

def get_model(model_path):
    if 'xgb' in model_path:
        xgb_model = XGBClassifier()
        xgb_model.load_model(model_path)
        model = xgb_model
    else: 
        model = get_obj(model_path)
    return model 

xbg = get_model('Models/xgb_v1.pkl') # an xgb

tree = model = get_model('Models/dt_v1.pkl') # a decition tree