How to read .pkl file to a tokenizer object?

Question

I use

 pickle.dump(tokenizer, open(TOKENIZER_MODEL, "wb"), protocol=0)

to generate a.pkl file, when I trying to read it using

pickle.load(open('tokenizer.pkl'))

it raises

TypeError: a bytes-like object is required, not 'str'

but when I trying to change this object to bytes-like object using

tkr = open('tokenizer.pkl')
bytes(tkr,encoding='utf-8')

it raises

TypeError: encoding without a string argument

I was wodering that if there is anyway to read this.pkl file or io.TextIOWrapper file to a tokenizer object?

Answer 1

When you want to read the file you need to write

file = open('tokenizer.pkl', 'rb')    
pickle.load(file)

• 'r' stands for read mode
• 'b' stands for binary mode

Answer 2

Ok I've never used pickle but after trying it really quickly in repl.it I think that your problem is in this line

pickle.load(open('tokenizer.pkl'))

It should be

pickle.load(open('tokenizer.pkl','rb'))