Parsing Python Module Docstrings

Question

Is it possible to parse module-level docstrings with the AST?

I am working on a python documenter here and visiting the module tokens and grabbing the documentation does not yield the module-level docstring. So far, I've had to resort to importing the module and grabbing its __doc__ or using inspect to grab the documentation.

I looked into the pydoc module source for clues as to how other documenters parse docstrings, and discovered that pydoc ends up having to do basically the same thing as my documenter in order to grab the module-level strings.

Am I missing something? Is the only way to parse module-level docstrings through actually importing the module, or is it possible to parse the docstrings out of the AST directly?

Answer 1

Maybe I miss-understand the question, but can't you just do this (python 2.7.1)?

test file:

"""
DOC STRING!!
"""

def hello():
    'doc string'
    print 'hello'

hello()

Interactive session:

>>> M = ast.parse(''.join(open('test.py')))
>>> ast.get_docstring(M)
'DOC STRING!!'

You can also walk through the ast, looking for the slot the doc string would be in.

>>> M._fields
('body',)
>>> M.body
[<_ast.Expr object at 0x10e5ac710>, <_ast.FunctionDef object at 0x10e5ac790>, <_ast.Expr object at 0x10e5ac910>]
>>> # doc would be in the first slot
>>> M.body[0]._fields
('value',)
>>> M.body[0].value
<_ast.Str object at 0x10e5ac750>
>>> # it contains a string object, so maybe it's the doc string
>>> M.body[0].value._fields
('s',)
>>> M.body[0].value.s
'\nDOC STRING!!\n'