What difference between pickle and _pickle in python 3?

Question

I am new in python and want implement fast object serialization. I was trying to use json, but it was too slow, also was trying to use marshall module, but the size of the objects serialized by marshall 6-7 times more than pickle, so i decided to use pickle in my project. I read about cPickle module, read that it quite fast, but there is no such module in python 3 and docs says that module named _pickle is written in C. So in my projects i use

import _pickle as pickle

Is any difference between pickle and _pickle? How i can implement faster objects serialization/deserialization?

Answer 1

The pickle module already imports _pickle if available. It is the C-optimized version of the pickle module, and is used transparently.

From the pickle.py source code :

# Use the faster _pickle if possible
try:
    from _pickle import *
except ImportError:
    Pickler, Unpickler = _Pickler, _Unpickler

and from the pickle module documentation :

The pickle module has an transparent optimizer ( _pickle ) written in C. It is used whenever available. Otherwise the pure Python implementation is used.

In Python 2, _pickle was known as cPickle , but has been updated to allow the transparent use as an implementation detail.

Answer 2

From the Library Changes section of the What's New In Python 3.0 documentation:

A common pattern in Python 2.x is to have one version of a module implemented in pure Python, with an optional accelerated version implemented as a C extension; for example, pickle and cPickle . This places the burden of importing the accelerated version and falling back on the pure Python version on each user of these modules. In Python 3.0, the accelerated versions are considered implementation details of the pure Python versions. Users should always import the standard version, which attempts to import the accelerated version and falls back to the pure Python version. The pickle / cPickle pair received this treatment. The profile module is on the list for 3.1. The StringIO module has been turned into a class in the io module.

Since it is a python convention that implementation details are prepended with an underscore, cPickle became _pickle . Notably this means that if you are importing _pickle , the API has no guaranteed contract and could break backwards-compatibility in future releases of python3, as unlikely as that may be.