Python bdist and distribute packages for install without PyPi

Question

I'm developing a system that itself is not terribly complex: just a virtualenv with a collection of packages, each with their own dependencies. All told (with dependencies), about 30 packages need to be installed.

The following constraints complicate the setup somewhat:

• The internet is not guaranteed to be accessible. In fact we assume that PyPI is unavailable at the time of install. (A non-network debian stable repository is available, which allows for virtualenv and company)
• Some of the packages/dependencies have C extensions which are essential to the performance of the application.
• It is desired that the C extentions be compiled in advance to avoid installation of GCC and company.

Within these constraints, what is the fastest, the easiest or even (preferably) the 'right' way to package (and then install) the python module dependencies? Do I just grab the source for each package and bdist it myself?

Answer 1

This questions is a bit old, but in any case, since pip version 1.4 (2013-07-23), wheel has been supported as a binary distribution format with a richer interface and better support. For those finding this thread, consider using wheel instead of egg.

Example usage (from the wheel documentation):

To build wheels for your requirements and all their dependencies to a local directory:

 pip install wheel pip wheel --wheel-dir=/local/wheels -r requirements.txt 

And then to install those requirements just using your local directory of wheels (and not from PyPI):

 pip install --no-index --find-links=/local/wheels -r requirements.txt 

Answer 2

The pip --find-links option supports local directories. Simply put all your sdist and egg distributions into the one directory and pip will find them there.

Binary distributions, I'd build as eggs (with setup.py bdist_egg ), for easy installation without the need to compile.

Answer 3

The method I've settled on for now is this :

# To gather together all of the dependencies (ex: pymongo, ujson):
easy_install -zmaxd /my/packaging/directory pymongo>=2.3 ujson

# Secondarily, package my custom module
/mymodule/setup.py sdist
cp /mymodule/dist/mymodule.tar /my/packaging/directory/

Later, since most version of pip in the world don't support eggs, use easy_install to install as well. (Recent versions of pip do, I understand but not everyone is up to date):

easy_install -H None -f file:///installed/packaging/directory mymodule

The dependencies for mymodule Still seeing if this works, but it seems close to the 'right' way of doing things.

Thanks to Martijn for pointing me towards eggs.