UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Question

I am saving a list using pickle.dumps() as so:

my_list = ['Hello', 'I', 'Have', 'a', 'question', 'camión']
my_pickle = pickle.dumps(my_list)       

Once I have the pickle created I am uploading it to a container in Azure Batch:

blob_service.block_service.create_blob_from_bytes('containername', 'filename', my_pickle)

And getting it back:

my_bytes = blob_service.block_service.get_blob_to_bytes('containername', 'filename')

What I want ( my_list ) is inside my_bytes.content and, if I printed I get:

b'\\x80\\x03]q\\x00(X\\x05\\x00\\x00\\x00Helloq\\x01X\\x01\\x00\\x00\\x00Iq\\x02X\\x04\\x00\\x00\\x00Haveq\\x03X\\x01\\x00\\x00\\x00aq\\x04X\\x08\\x00\\x00\\x00questionq\\x05X\\x07\\x00\\x00\\x00cami\\xc3\\xb3nq\\x06e.'

To get my_bytes back to the list, I tried to decode it as follows:

my_bytes.decode('utf-8')

But I am getting the following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Other encoding, as latin-1 , it is not giving mistake but it is returning

'\\x80\\x03]q\\x00(X\\x05\\x00\\x00\\x00Helloq\\x01X\\x01\\x00\\x00\\x00Iq\\x02X\\x04\\x00\\x00\\x00Haveq\\x03X\\x01\\x00\\x00\\x00aq\\x04X\\x08\\x00\\x00\\x00questionq\\x05X\\x07\\x00\\x00\\x00camiónq\\x06e.'

rather than my_list .

I have searched on Internet to find out why this is happening but I couldn't find anything that helped me, so any suggestion or advice is more than welcome. I would like to get my_list from my_bytes . Please note that I am interested on how to decode my_bytes .

Answer 1

Use pickle.loads to reverse pickle.dumps :

>>> s = b'\x80\x03]q\x00(X\x05\x00\x00\x00Helloq\x01X\x01\x00\x00\x00Iq\x02X\x04\x00\x00\x00Haveq\x03X\x01\x00\x00\x00aq\x04X\x08\x00\x00\x00questionq\x05X\x07\x00\x00\x00cami\xc3\xb3nq\x06e.'
>>> import pickle
>>> pickle.loads(s)
['Hello', 'I', 'Have', 'a', 'question', 'camión']

Answer 2

What I finally did was converting my list to string as:

my_list=str(my_list)

And upload it to Blob Storage as:

BlockStorage('<account_name>', '<account_key>').block_service.create_blob_from_text('<container_name>', '<file_name>', my_list)

And finally, to download it:

my_file_as_str = BlockStorage('<account_name>', '<account_key>').block_service.get_blob_to_text('<container_name>', '<file_name>')

To get back my list I just needed to do now:

my_list = eval(my_file_as_str.content)