使用协议选项的cPickle文件

Question

I have silly question. Is possible to re-pickle(cPickle) an already cPickled file using another protocol option, and also which protocol would be best for a very large file. Would appreciate any help/suggestions.

Answer 1

I don't really understand what you mean by pickling a file. Do you want to read a pickled file as binary without unpickling and then pickle it again? This should not be a problem but I do not see what good it would do...

The default protocol version in pickle is 0, ASCII ( pickle docs ). The latest binary protocol is 2, which gives you smaller file sizes. For me a binary pickle output was only half as big for som jpg picture as with an ASCII pickle, code below.

You might want to consider cerealizer which has a pickle like interface but is more secure. It seems to have its own binary protocol, file size being about as small as with pickle binary.

import cerealizer
#import cPickle as cerealizer

def save(data, filename):
    f = open(filename,"wb")
    cerealizer.dump(data, f, protocol=2)
    f.close()

def load(filename):
    f = open(filename,"rb")
    p = cerealizer.load(f)
    f.close()
    return(p)

if __name__ == "__main__":
    import PIL.Image
    import cStringIO as StringIO

    stringIO = StringIO.StringIO()
    im = PIL.Image.open("picture.jpg")
    im.save(stringIO, "JPEG")

    stringIO.seek(0)
    save(stringIO.read(), "testCerealizerIm.txt")

    binaryImageData = load("testCerealizerIm.txt")

    stringIO2 = StringIO.StringIO()
    stringIO2.write(binaryImageData)

    stringIO2.seek(0)
    im = PIL.Image.open(stringIO2)
    im.show()

Answer 2

You unpickle it and repickle it. The protocol version can be choosen as part of the pickle API:

http://docs.python.org/library/pickle.html