Python3 unpickle a string representation of bytes object

Question

Is there a good way to load a bytes object that is represented as a string, so it can be unpickled?

Basic Example

Here is a dumb example:

import pickle

mydict = { 'a': 1111, 'b': 2222 }
string_of_bytes_obj = str(pickle.dumps(mydict)) # Deliberate string representation for this quick example.

unpickled_dict = pickle.loads(string_of_bytes_obj) # ERROR!  Loads takes bytes-like object and not string.

Attempt at a Solution

One solution is of course to eval the string:

unpickled_dict = pickle.loads(eval(string_of_bytes_obj))

But, seems wrong to eval , especially when the strings might be coming over a network or from a file.

...

Any suggestions for a better solution?

Thanks!

Answer 1

For a safety concern you can use ast.literal_eval instead of eval :

>>> import ast
>>> pickle.loads(ast.literal_eval(string_of_bytes_obj))
{'b': 2222, 'a': 1111}

Answer 2

Is there a reason you need to have it as a str? If you're just writing it to file, you can 'wb' instead of 'w'. ( https://pythontips.com/2013/08/02/what-is-pickle-in-python/ )

import pickle

mydict = { 'a': 1111, 'b': 2222 }
dumped = pickle.dumps(mydict)
string_of_bytes_obj = str(dumped) # Deliberate string representation for this quick example. 

unpickled_dict = pickle.loads(dumped) 

Answer 3

You can use encoding="latin1" as an argument to str and then use bytes to convert back:

import pickle

mydict = { 'a': 1111, 'b': 2222 }
string_of_bytes_obj = str(pickle.dumps(mydict), encoding="latin1")

unpickled_dict = pickle.loads(bytes(string_of_bytes_obj, "latin1"))

Output:

>>> print(unpickled_dict)
{'a': 1111, 'b': 2222}

Answer 4

First of all i wouldn't use pickles to serialize data. instead use Json.

my solution with pickles

import pickle

mydict = { 'a': 1111, 'b': 2222 }
string_of_bytes_obj = pickle.dumps(mydict) # Deliberate string representation for this quick example.
print(string_of_bytes_obj)
unpickled_dict = pickle.loads(string_of_bytes_obj)
print(unpickled_dict)

BUT with json

import json

mydict = { 'a': 1111, 'b': 2222 }
string_of_bytes_obj = json.dumps(mydict) 
print(string_of_bytes_obj)
unpickled_dict = json.loads(string_of_bytes_obj)
print(unpickled_dict)

I highly recommend you to use json to serialize your data