Is `pickle.dump(d, f)` equivalent to `f.write(pickle.dumps(d))`?

Question

Given an arbitrary picklable Python data structure data , is

with open('a', 'bw') as f:
    f.write(pickle.dumps(data))

equivalent to

with open('a', 'bw') as f:
    pickle.dump(data, f)

ie can I assume this equivalence when writing code? Could you describe how you reached to this conclusion?

The use-case is to separate serialization from writing, since I may want to write pickle to a non-disk location.

Answer 1

Yes, they are equivalent. Looking at the source code of pickle.py :

def _dump(obj, file, protocol=None, *, fix_imports=True):
    _Pickler(file, protocol, fix_imports=fix_imports).dump(obj)

def _dumps(obj, protocol=None, *, fix_imports=True):
    f = io.BytesIO()
    _Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
    res = f.getvalue()
    assert isinstance(res, bytes_types)
    return res

dumps does the exact same thing as dump , just with an io.BytesIO object instead of a file object. It calls the internal _Pickler().dump() in the same way and simply returns the contents of the io.BytesIO object. Therefore, all f.write(pickle.dumps(data)) does is first forward the result to an io.BytesIO object then to the actual file instead of writing to the file directly.

Answer 2

They are not equivalent if an exception occurs during pickling. f.write(pickle.dumps(data)) will not write anything to the file. Whereas pickle.dump(data, f) will end up with a snapped or partial pickle.

Answer 3

Yes.

The documentation of dumps is pretty clear:

Return the pickled representation of the object as a bytes object, instead of writing it to a file.

Writing the same representation to a file will be the same as calling pickle.dump directly.

Also, "file" here means anything with a write method, from documentation of dump :

The file argument must have a write() method that accepts a single bytes argument. It can thus be an on-disk file opened for binary writing, an io.BytesIO instance, or any other custom object that meets this interface.