Python Packaging multiple subpackages with different data directories

Question

I have a structure of the directory as such with foobar and alphabet data directories together with the code something.py :

\mylibrary
    \packages
         \foobar
             foo.zip
             bar.zip
         \alphabet
             abc.zip
             xyz.zip
          something.py
     setup.py

And the goal is such that users can pip install the module as such:

pip install mylibrary[alphabet]

And that'll only include the data from the packages/alphabet/* data and the python code. Similar behavior should be available for pip install mylibrary[foobar] .

If the user installs without the specification:

pip install mylibrary

Then it'll include all the data directories under packages/ .

Currently, I've tried writing the setup.py with Python3.5 as such:

import glob
from setuptools import setup, find_packages


setup(
  name = 'mylibrary',
  packages = ['packages'],
  package_data={'packages':glob.glob('packages' + '/**/*.txt', recursive=True)},
)

That will create a distribution with all the data directories when users do pip install mylibrary .

How should I change the setup.py such that specific pip installs like pip install mylibrary[alphabet] is possible?

Answer 1

Firs you have to package and publish alphabet and foobar as a separate packages as pip install mylibrary[alphabet] means

pip install mylibrary
pip install alphabet

After that add alphabet and foobar as extras :

setup(
    …,
    extras = {
        'alphabet': ['alphabet'],
        'foobar': ['foobar'],
    }
)

The keys in the dictionary are the names used in pip install mylibrary[EXTRA_NAME] , the values are a list of package names that will be installed from PyPI.

PS. And no, you cannot use extras to install some data files that are not available as packages from PyPI.