I have built Xgb model with xgboost package in python. I saved the model using pickle and joblib which works perfectly in my windows 10 system. But it is giving error in AWS instance I am trying to run. The error seems to be related to serialization.
>>> import joblib
>>> joblib.load(xgb_low_lr_fin.sav)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'xgb_low_lr_fin' is not defined
>>> joblib.load("xgb_low_lr_fin.sav")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/joblib/numpy_pickle.py", line 598, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/joblib/numpy_pickle.py", line 526, in _unpickle
obj = unpickler.load()
File "/home/ubuntu/anaconda3/lib/python3.7/pickle.py", line 1085, in load
dispatch[key[0]](self)
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/joblib/numpy_pickle.py", line 339, in load_build
Unpickler.load_build(self)
File "/home/ubuntu/anaconda3/lib/python3.7/pickle.py", line 1549, in load_build
setstate(state)
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/xgboost-1.0.0_SNAPSHOT-py3.7.egg/xgboost/core.py", line 1132, in __setstate__
_LIB.XGBoosterUnserializeFromBuffer(handle, ptr, length))
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/xgboost-1.0.0_SNAPSHOT-py3.7.egg/xgboost/core.py", line 189, in _check_call
raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [10:43:02] src/learner.cc:660: Check failed: header == serialisation_header_ (
The above code works perfectly in windows and in mac os.
I had a similar problem when trying to load a pickle file that was pickled on a different machine, and it was related to pickle serialization being different in Python 2 and Python 3 - or maybe even in the different versions of Pickle itself.
Try and check you pickle and python versions and make them match :)
In my case this error was caused by my xgboost versions being slightly different. My model was trained with a CPU implementation of xgboost. After building and installing xgboost with GPU support, the model that was trained on CPU no longer loaded. Re-installing the normal version of xgboost from pypi seemed to resolve my issue.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.