How do you get a list of pydoc accessible paths for a python package, e.g. numpy or tensorflow?

Question

I am trying to get all the paths in a python package (for example numpy) that you can use pydoc on. Ideally, this list would contain elements like:

numpy numpy.linalg numpy.linalg.det ...

So, ideally one would recurse into submodules and classes inside the package, and these paths would be what tab completion would give you in an enabled editor. I would prefer to get this list from the installed packages for consistency.

The purpose of this list is to feed it to a fuzzy completion engine to select items to then send to pydoc, eg. by typing "det", the engine would select all entries containing that string (so I don't have to remember the full path to it), and I could choose the one I want from there. That part is easy, once I have the list of paths! It doesn't matter if this is slow, I only need to do it once and can save it to file for use later. It also doesn't matter if the list is very long. The completion engine can handle very large lists.

Is there a straightforward way to do this? I have tried a few variations with the inspect module, recursing through the dict attributes, even one with rlcompleter, but so far I have either not had much success with the recursion, or the paths are not useful in pydoc.

Answer 1

Yes, there is a way to do just that. For example, for NumPy:

from pkgutil import walk_packages
import numpy

# Obtain list with all modules found in the package
mod_list = [name for _, name, _ in walk_packages(numpy.__path__,
                                                 numpy.__name__+'.')]

This will give you a list mod_list that contains all submodules that the NumPy package has, given in strings.

Answer 2

Oh, from your comment, you mean all type s that pydoc traverses?

Pydoc does limit it's documentation to the __all__ list, but I'm pretty sure that if __all__ equaled one class, it would still document the class's methods, etc, which are types in their own right.

    import numpy
    print(numpy.__all__)

I would try to use pydoc's type traversal code, but it so mixed up with UI concerns that it doesn't really work well as a "better" inspect module. Short of forking pydoc and modifying the type traversal code until you can create a "type" accumulator, I'm not sure this is possible.