酱菜堆没有唯一的代表？ 创建Python字典的唯一序列化

Question

I'm writing a caching script, which takes a certain object and stores it as a file. The object's initial parameters are supposed to be used as filename (because I want different caches for different initial parameters).

I thought serialization was a bijection, so my plan was to serialize the (parameter) dictionary, create a hex using hashlib.sha224().hexdigest() and use the first XX characters as a filename. However, repeatedly serializing the dictionary using pickle.dumps yields different serializations...

I want to serialize a dictionary:

attr = {'I': 1,
 'LBar': 1,
 'N': 50,
 'ProdMatch': 1,
 'T': 10,
 'alpha': 0.5,
 'b': 0.1,
 'c': 0.1,
 'delta': 0.1,
 'deltaN': 0.02,
 'deltaT': 10.0,
 'logspace': False,
 'nT': 1,
 'period': 'quarterly',
 'rho': 0.03,
 'sigma': 0.5}

ser = pickle.dumps(attr)
print(pickle.loads(ser))

Which gives me as output the input dictionary and the following string.

b'\x80\x03}q\x00(X\x01\x00\x00\x00cq\x01G?\xb9\x99\x99\x99\x99\x99\x9aX\x01\x00\x00\x00Nq\x02K2X\x01\x00\x00\x00Tq\x03K\nX\x05\x00\x00\x00deltaq\x04G?\xb9\x99\x99\x99\x99\x99\x9aX\x06\x00\x00\x00deltaNq\x05G?\x94z\xe1G\xae\x14{X\x06\x00\x00\x00deltaTq\x06G@$\x00\x00\x00\x00\x00\x00X\x05\x00\x00\x00alphaq\x07G?\xe0\x00\x00\x00\x00\x00\x00X\x03\x00\x00\x00rhoq\x08G?\x9e\xb8Q\xeb\x85\x1e\xb8X\x04\x00\x00\x00LBarq\tK\x01X\x01\x00\x00\x00bq\nG?\xb9\x99\x99\x99\x99\x99\x9aX\x06\x00\x00\x00periodq\x0bX\t\x00\x00\x00quarterlyq\x0cX\x02\x00\x00\x00nTq\rK\x01X\x01\x00\x00\x00Iq\x0eK\x01X\t\x00\x00\x00ProdMatchq\x0fK\x01X\x05\x00\x00\x00sigmaq\x10G?\xe0\x00\x00\x00\x00\x00\x00X\x08\x00\x00\x00logspaceq\x11\x89u.'

I then restart my Python interpreter and run the same script:

b'\x80\x03}q\x00(X\x05\x00\x00\x00deltaq\x01G?\xb9\x99\x99\x99\x99\x99\x9aX\x05\x00\x00\x00alphaq\x02G?\xe0\x00\x00\x00\x00\x00\x00X\x04\x00\x00\x00LBarq\x03K\x01X\x01\x00\x00\x00Iq\x04K\x01X\x02\x00\x00\x00nTq\x05K\x01X\x01\x00\x00\x00bq\x06G?\xb9\x99\x99\x99\x99\x99\x9aX\x01\x00\x00\x00Tq\x07K\nX\x03\x00\x00\x00rhoq\x08G?\x9e\xb8Q\xeb\x85\x1e\xb8X\x01\x00\x00\x00cq\tG?\xb9\x99\x99\x99\x99\x99\x9aX\t\x00\x00\x00ProdMatchq\nK\x01X\x06\x00\x00\x00periodq\x0bX\t\x00\x00\x00quarterlyq\x0cX\x06\x00\x00\x00deltaNq\rG?\x94z\xe1G\xae\x14{X\x06\x00\x00\x00deltaTq\x0eG@$\x00\x00\x00\x00\x00\x00X\x05\x00\x00\x00sigmaq\x0fG?\xe0\x00\x00\x00\x00\x00\x00X\x08\x00\x00\x00logspaceq\x10\x89X\x01\x00\x00\x00Nq\x11K2u.'

These are different serializations. Pickle is not broken, as loads() will regenerate the initial dictionary, but I can't have this behavior for my purposes, as it will not find a stored file if it searches under a different hash. I also tried dill.dumps , with the same result.

Answer 1

Dictionaries prior to Python 3.6 don't have a specific, repeatable order. In fact, in Python 3 up to Python 3.5 their order is randomize.

Try sorting the items:

import pickle
ser = pickle.dumps(sorted(attr.items()))
print(dict(pickle.loads(ser)))