在 Python 3.8 中加载使用 Python 2.7 创建的 pickle 文件时出错

Question

I have a pickle file which contains floating-point values. This file was created with Python 2.7. In Python 2.7 I used to load it like:

matrix_file = pickle.load(open('matrix.pickle', 'r'))

Now in Python 3.8 this code is giving error

TypeError: a bytes-like object is required, not 'str'

When I trid with 'rb' I got this error

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 8: ordinal not in range(128)

So I tried another method

matrix_file = pickle.load(open('matrix.pickle', 'r', encoding='utf-8'))

Now I get a different error

TypeError: a bytes-like object is required, not 'str'

Update : When I try loading with joblib, I get this error

ValueError: You may be trying to read with python 3 a joblib pickle generated with python 2. This feature is not supported by joblib.

Answer 1

The file must be opened in binary mode and you need to provide an encoding for the pickle.load call. Typically, the encoding should either be "latin-1" (for pickles with numpy arrays, datetime , date and time objects, or when the strings were logically Latin-1), or "bytes" (to decode Python 2 str as bytes objects). So the code should be something like:

with open('matrix.pickle', 'rb') as f:
    matrix_file = pickle.load(f, encoding='latin-1')

This assumes it was originally containing numpy arrays; if not, "bytes" might be the more appropriate encoding. I also used a with statement just for good form (and to ensure deterministic file closing on non-CPython interpreters).