將復雜字典保存到文件的最佳方法是什么？

Question

我有一本包含元組鍵和numpy數組值的字典。 我嘗試使用h5和pickle保存它，但收到錯誤消息。 將對象保存到文件的最佳方法是什么？

import numpy as np
from collections import defaultdict
Q =defaultdict(lambda: np.zeros(2))
Q[(1,2,False)] = np.array([1,2])
Q[(1,3,True)] = np.array([3,4])

>>> Q
defaultdict(<function <lambda> at 0x10c51ce18>, {(1, 2, False): array([1, 2]), (1, 3, True): array([3, 4])})

np.save追溯：

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-99-a071e1561501> in <module>()
----> 1 np.save('Q.npy', Q)

~/anaconda3_420/lib/python3.5/site-packages/numpy/lib/npyio.py in save(file, arr, allow_pickle, fix_imports)
    509         arr = np.asanyarray(arr)
    510         format.write_array(fid, arr, allow_pickle=allow_pickle,
--> 511                            pickle_kwargs=pickle_kwargs)
    512     finally:
    513         if own_fid:

~/anaconda3_420/lib/python3.5/site-packages/numpy/lib/format.py in write_array(fp, array, version, allow_pickle, pickle_kwargs)
    584         if pickle_kwargs is None:
    585             pickle_kwargs = {}
--> 586         pickle.dump(array, fp, protocol=2, **pickle_kwargs)
    587     elif array.flags.f_contiguous and not array.flags.c_contiguous:
    588         if isfileobj(fp):

AttributeError: Can't pickle local object 'mc_control_epsilon_greedy.<locals>.<lambda>'

Answer 1

將其另存為普通字典怎么樣？ 保存期間不需要defaultdict行為。

In [126]: from collections import defaultdict
In [127]: Q =defaultdict(lambda: np.zeros(2))
     ...: Q[(1,2,False)] = np.array([1,2])
     ...: Q[(1,3,True)] = np.array([3,4])
     ...: Q[(3,4,False)]
     ...: 
Out[127]: array([0., 0.])
In [128]: Q
Out[128]: 
defaultdict(<function __main__.<lambda>>,
            {(1, 2, False): array([1, 2]),
             (1, 3, True): array([3, 4]),
             (3, 4, False): array([0., 0.])})

我們可以通過以下方式將其退出defaultdict包裝：

In [130]: dict(Q)
Out[130]: 
{(1, 2, False): array([1, 2]),
 (1, 3, True): array([3, 4]),
 (3, 4, False): array([0., 0.])}

然后我們可以腌制它（我使用np.save作為腌制快捷方式）

In [131]: np.save('stack49963862', np.array(dict(Q)))

load給出一個包含此字典的對象數組：

In [132]: P = np.load('stack49963862.npy')
In [133]: P
Out[133]: 
array({(1, 2, False): array([1, 2]), (1, 3, True): array([3, 4]), (3, 4, False): array([0., 0.])},
      dtype=object)

In [138]: P.item()
Out[138]: 
{(1, 2, False): array([1, 2]),
 (1, 3, True): array([3, 4]),
 (3, 4, False): array([0., 0.])}

我們可以通過更新輕松地重新創建defaultdict：

In [134]: Q1 =defaultdict(lambda: np.zeros(2))
In [139]: Q1.update(P.item())
In [140]: Q1
Out[140]: 
defaultdict(<function __main__.<lambda>>,
            {(1, 2, False): array([1, 2]),
             (1, 3, True): array([3, 4]),
             (3, 4, False): array([0., 0.])})

Answer 2

pickle沒問題

import pickle
import numpy as np

x = {(1,2,False): np.array([1,4]), (1,3,False): np.array([4,5])}

with open('filename.pickle', 'wb') as handle:
    pickle.dump(x, handle, protocol=pickle.HIGHEST_PROTOCOL)
with open('filename.pickle', 'rb') as handle:
    y = pickle.load(handle)

print x
print y

---

編輯后：

您實際擁有的是lambda ，默認情況下無法腌制。 您需要安裝dill並將其導入才能正常工作（請參見此答案 ）

它應該是這樣的：

import pickle
import numpy as np
from collections import defaultdict
import dill # doesn't come with default anaconda. Install with "conda install dill"

x = defaultdict(lambda: np.zeros(2))
with open('filename.pickle', 'wb') as handle:
    pickle.dump(x, handle, protocol=pickle.HIGHEST_PROTOCOL)
with open('filename.pickle', 'rb') as handle:
    y = pickle.load(handle)

print x
print y

輸出：

# no errors :-)
defaultdict(<function <lambda> at 0x000000000CD0C898>, {})
defaultdict(<function <lambda> at 0x0000000002614C88>, {})

---

OP的解決方案：您編輯過的解決方案仍然為我產生了相同的錯誤，但是效果很好：

import pickle
import dill
dill_file = open("Q.pickle", "wb")
dill_file.write(dill.dumps(Q))
dill_file.close()

在我的機器（使用Spyder的Win 8.1 64位系統）上，使用簡單dill時沒有錯誤。