[英]How can I use pickle to save a dict (or any other Python object)?
I have looked through the information that the Python docs give, but I'm still a little confused.我已经查看了Python 文档提供的信息,但我仍然有点困惑。 Could somebody post sample code that would write a new file then use pickle to dump a dictionary into it?
有人可以发布编写新文件然后使用 pickle 将字典转储到其中的示例代码吗?
Try this:尝试这个:
import pickle
a = {'hello': 'world'}
with open('filename.pickle', 'wb') as handle:
pickle.dump(a, handle, protocol=pickle.HIGHEST_PROTOCOL)
with open('filename.pickle', 'rb') as handle:
b = pickle.load(handle)
print a == b
import pickle
your_data = {'foo': 'bar'}
# Store data (serialize)
with open('filename.pickle', 'wb') as handle:
pickle.dump(your_data, handle, protocol=pickle.HIGHEST_PROTOCOL)
# Load data (deserialize)
with open('filename.pickle', 'rb') as handle:
unserialized_data = pickle.load(handle)
print(your_data == unserialized_data)
The advantage of HIGHEST_PROTOCOL
is that files get smaller. HIGHEST_PROTOCOL
的优点是文件变小了。 This makes unpickling sometimes much faster.这使得解酸有时更快。
Important notice : The maximum file size of pickle is about 2GB.重要提示:pickle 的最大文件大小约为 2GB。
import mpu
your_data = {'foo': 'bar'}
mpu.io.write('filename.pickle', data)
unserialized_data = mpu.io.read('filename.pickle')
For your application, the following might be important:对于您的应用程序,以下内容可能很重要:
See also: Comparison of data serialization formats另请参阅: 数据序列化格式的比较
In case you are rather looking for a way to make configuration files, you might want to read my short article Configuration files in Python如果您正在寻找一种制作配置文件的方法,您可能需要阅读我的短文Python 中的配置文件
# Save a dictionary into a pickle file.
import pickle
favorite_color = {"lion": "yellow", "kitty": "red"} # create a dictionary
pickle.dump(favorite_color, open("save.p", "wb")) # save it into a file named save.p
# -------------------------------------------------------------
# Load the dictionary back from the pickle file.
import pickle
favorite_color = pickle.load(open("save.p", "rb"))
# favorite_color is now {"lion": "yellow", "kitty": "red"}
In general, pickling a dict
will fail unless you have only simple objects in it, like strings and integers.一般来说,pickling
dict
会失败,除非其中只有简单的对象,例如字符串和整数。
Python 2.7.9 (default, Dec 11 2014, 01:21:43)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import *
>>> type(globals())
<type 'dict'>
>>> import pickle
>>> pik = pickle.dumps(globals())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle module objects
>>>
Even a really simple dict
will often fail.即使是一个非常简单的
dict
也经常会失败。 It just depends on the contents.这仅取决于内容。
>>> d = {'x': lambda x:x}
>>> pik = pickle.dumps(d)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <function <lambda> at 0x102178668>: it's not found as __main__.<lambda>
However, if you use a better serializer like dill
or cloudpickle
, then most dictionaries can be pickled:但是,如果您使用更好的序列化程序,例如
dill
或cloudpickle
,那么大多数字典都可以被pickle:
>>> import dill
>>> pik = dill.dumps(d)
Or if you want to save your dict
to a file...或者,如果您想将
dict
保存到文件中...
>>> with open('save.pik', 'w') as f:
... dill.dump(globals(), f)
...
The latter example is identical to any of the other good answers posted here (which aside from neglecting the picklability of the contents of the dict
are good).后一个示例与此处发布的任何其他好答案相同(除了忽略
dict
内容的可挑选性之外还不错)。
Simple way to dump a Python data (eg dictionary) to a pickle file.将 Python 数据(例如字典)转储到 pickle 文件的简单方法。
import pickle
your_dictionary = {}
pickle.dump(your_dictionary, open('pickle_file_name.p', 'wb'))
>>> import pickle
>>> with open("/tmp/picklefile", "wb") as f:
... pickle.dump({}, f)
...
normally it's preferable to use the cPickle implementation通常最好使用 cPickle 实现
>>> import cPickle as pickle
>>> help(pickle.dump)
Help on built-in function dump in module cPickle:
dump(...)
dump(obj, file, protocol=0) -- Write an object in pickle format to the given file.
See the Pickler docstring for the meaning of optional argument proto.
If you just want to store the dict in a single file, use pickle
like that如果您只想将 dict 存储在单个文件中,请像这样使用
pickle
import pickle
a = {'hello': 'world'}
with open('filename.pickle', 'wb') as handle:
pickle.dump(a, handle)
with open('filename.pickle', 'rb') as handle:
b = pickle.load(handle)
If you want to save and restore multiple dictionaries in multiple files for caching and store more complex data, use anycache .如果要在多个文件中保存和恢复多个字典以缓存和存储更复杂的数据,请使用anycache 。 It does all the other stuff you need around
pickle
它可以完成您在
pickle
周围所需的所有其他东西
from anycache import anycache
@anycache(cachedir='path/to/files')
def myfunc(hello):
return {'hello', hello}
Anycache stores the different myfunc
results depending on the arguments to different files in cachedir
and reloads them. Anycache 根据
cachedir
不同文件的参数存储不同的myfunc
结果并重新加载它们。
See the documentation for any further details.有关更多详细信息,请参阅文档。
import pickle
dictobj = {'Jack' : 123, 'John' : 456}
filename = "/foldername/filestore"
fileobj = open(filename, 'wb')
pickle.dump(dictobj, fileobj)
fileobj.close()
FYI, Pandas has a method to save pickles now.仅供参考,熊猫现在有一种保存泡菜的方法。
I find it easier.我觉得比较容易。
pd.to_pickle(object_to_save,'/temp/saved_pkl.pickle' )
If you want to handle writing or reading in one line without file opening:如果您想在不打开文件的情况下在一行中处理写入或读取:
import joblib
my_dict = {'hello': 'world'}
joblib.dump(my_dict, "my_dict.pickle") # write pickle file
my_dict_loaded = joblib.load("my_dict.pickle") # read pickle file
I've found pickling confusing (possibly because I'm thick).我发现酸洗令人困惑(可能是因为我很厚)。 I found that this works, though:
不过,我发现这有效:
myDictionaryString=str(myDictionary)
Which you can then write to a text file.然后您可以将其写入文本文件。 I gave up trying to use pickle as I was getting errors telling me to write integers to a .dat file.
我放弃了尝试使用 pickle,因为我收到错误提示我将整数写入 .dat 文件。 I apologise for not using pickle.
我为没有使用泡菜而道歉。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.