pydoc creates incomplete documentation

Question

I created my first package, and I noticed that the documentation that shows up when a user calls help(my_package) is incomplete. I'd would be nice to get the classes, methods, functions, and attributes listed when calling help() . And ideally the description of the functions from the docstring s.

Reading the pydoc documentation, I still have no clue how to do it, since the information provided there is a little bit sparse.

When I understood correctly, the "output" for help() is automatically created when a user calls this function? Or do I have to provide and specify something extra?

Currently, it looks like this when I call help() on my package (here: pdbsr):

Help on package pdbsr:

NAME
    pdbsr

FILE
    /.../pdbsr/__init__.py

PACKAGE CONTENTS
    bugtest (package)
    exceptions (package)
    extras (package)
    info (package)
    pdbfile (package)

SUBMODULES
    pdb_properties
    slide

DATA
    __version__ = '0.1.0'
    l2lvl = ['HEADER    LANTIBIOTIC-BINDING-PROTEIN             06-JUL-12 ...
    l3eiy = ['HEADER    HYDROLASE                               17-SEP-08 ...
    s2lvl = 'HEADER    LANTIBIOTIC-BINDING-PROTEIN           ...    0    0...
    s3eiy = 'HEADER    HYDROLASE                             ...    0   13...

VERSION
    0.1.0

And when I call the submodules, eg, pdbsr.exceptions:

Help on package pdbsr.exceptions in pdbsr:

NAME
    pdbsr.exceptions

FILE
    /.../pdbsr/exceptions/__init__.py

PACKAGE CONTENTS
    pdb_exceptions

(END) 

Here is an overview over my current folder structure:

And my setup file currently looks like this:

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup



setup(
      name='pdbsr',
      version='0.1.0',
      description='Protein Structure File Utilities',
      long_description=open('README.rst').read() + '\n\n' +
                       open('HISTORY.rst').read(),
      author='Sebastian Raschka',
      author_email='...',
      license=open('LICENSE').read(),
      #url='...',
      packages = [
         'pdbsr',
         'pdbsr.bugtest', 
         'pdbsr.exceptions', 
         'pdbsr.pdbfile',
         'pdbsr.extras',
         'pdbsr.info'
      ],
      package_dir={'pdbsr': 'pdbsr'},
      package_data={'': ['LICENCE']},
      install_requires=[''],
      include_package_data=True,
      )

And this is the content of my uppermost __init__.py file:

from info.version import __version__

from pdbfile.new_pdb import *
from pdbfile.load_pdb import *
from pdbfile.pdb_lig import *
from pdbfile.pdb_prot import *

from bugtest.doct_2lvl import *
from bugtest.doct_3eiy import *

import extras.slide_functions as slide

Answer 1

The issue really is that pydoc has to be run on the module first and then each py file within the module. I am using windows so I added a SET PYTHONPATH=C:/Python27 to a CMD file. bde is my package put into site-packages, and inside that there are a number of scripts.

%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde

When you run pydoc on the module name you will get a .html in your current directory which has a summary from the init .py file and links to all the included files. But they need a target for the links to work.

So just run pydoc again on each submodule:

%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.auto_model 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.settings 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.static 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.validate_csv
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.model 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.mixin 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.create_table 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.make_model 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.apply 
%PYTHONPATH%/python %PYTHONPATH%\lib\pydoc.py -w bde.copy_data

Then you will have a complete html tree with all the files referenced. And so on if you have sub-sub modules. If you have some built-in modules like sys they will not link, but you can run pydoc on them as well.

Answer 2

Each of your classes and functions, both in and outside of classes, needs to have a docstring * in the code *as below, then help will do it's magic:

class SomeClass():
   """ Documentation at a high level about the class """

   def ClassMethod():
      """ Documentation about class method """
      # Actual Code

Copied from Comments:

the thing is that all of my classes and methods already HAVE docstrings as you described. Eg, when I call help(pdbsr.NewPdb) everything is fine for this class, but the Class does not show up in the general help(pdbsr) - this is what bothers me – Sebastian Raschka

Have you tried having an __all__ = ["package_item", ] in __init__.py files rather than from ???? import * statements?

yes, I tried it first and really couldn't get it to work with all = [...] - I don't know why. My goal is that all classes and functions should be available as eg, pdbsr.function() without using more than 1 dot notation for the sub-packages. How would you do that with all considering my folder structure? I also tried to use an all in the __init__ files of the subpackages, but for some reason I couldnt get it to work :(

I think - from looking at help out put you need to have in each of the submodules __init__.py an '__all__ = [...]' where the ... is replaced with a comma separated list of quoted names of what it include from that module in the case of a 'from module import *' - when I first started using modules I missed that it was a list of names not a list of objects.