Is there an easier way to package with Python?

Question

I tried to package a django app today. It's a big baby, and with the setup file, I have to manually write all packages and sub packages in the 'package' parameter. Then I have to find a way to copy fixtures, htmls / Css / image files, documentations, etc.

It's a terrible way to work. We are computer scientists, we automatize, doing this makes no sense.

And what when I change my app structure ? I have to rewrite the setup.py.

Is there a better way ? Some tool to automate that ? I can't believe a language than value developer time like Python makes packaging such a chore.

I want to be able to eventually install the app using a simple pip install. I know about build out, but it's not much simpler, and is not pip friendly.

Answer 1

At the very least if you use setuptools (an alternative to the stdlib's distutils ) you get an awesome function called find_packages() which when ran from the package root returns a list of package names in dot-notation suitable for the packages parameter.

Here is an example:

# setup.py

from setuptools import find_packages, setup

setup(
    #...
    packages=find_packages(exclude='tests'),
    #...
)

ps Packaging sucks in every language and every system. It sucks no matter how you slice it.

Answer 2

I've been through this pain myself today. I used the following, yoinked straight from Django's setup.py , which walks the app's filesystem looking for packages and data files (assuming you never mix the two):

import os
from distutils.command.install import INSTALL_SCHEMES

def fullsplit(path, result=None):
    """
    Split a pathname into components (the opposite of os.path.join) in a
    platform-neutral way.
    """
    if result is None:
        result = []
    head, tail = os.path.split(path)
    if head == '':
        return [tail] + result
    if head == path:
        return result
    return fullsplit(head, [tail] + result)

# Tell distutils to put the data_files in platform-specific installation
# locations. See here for an explanation:
# http://groups.google.com/group/comp.lang.python/browse_thread/thread/35ec7b2fed36eaec/2105ee4d9e8042cb
for scheme in INSTALL_SCHEMES.values():
    scheme['data'] = scheme['purelib']

# Compile the list of packages available, because distutils doesn't have
# an easy way to do this.
packages, data_files = [], []
root_dir = os.path.dirname(__file__)
if root_dir != '':
    os.chdir(root_dir)
myapp_dir = 'myapp'

for dirpath, dirnames, filenames in os.walk(myapp_dir):
    # Ignore dirnames that start with '.'
    for i, dirname in enumerate(dirnames):
        if dirname.startswith('.'): del dirnames[i]
    if '__init__.py' in filenames:
        packages.append('.'.join(fullsplit(dirpath)))
    elif filenames:
        data_files.append([dirpath, [os.path.join(dirpath, f) for f in filenames]])

Answer 3

I think the tool you are looking for is Buildout . There lots of places where you can learn more about it, from SlideShare to Pycon videos .

Other similar or related tools which you might want to check out include virtualenv, Fabric, and PIP .

Answer 4

I've been doing a bit of research myself on Django deployment methods recently.

I found these two resources very useful:

• http://www.caktusgroup.com/blog/2010/04/22/basic-django-deployment-with-virtualenv-fabric-pip-and-rsync/ - A basic overview of how to use virtualenv, pip and fabric to do easy Django deployment from dev machine to production.
• https://github.com/jacobian/django-deployment-workshop/ - Jacob(ian) did a Django deployment workshop, with all the code posted on GitHub. All the necessary links are there as well.