How to search for a __main__ module using pyclbr in Python3?

Question

I want to get all the functions and classes in module: __main__ of the source code directory: /tmp/rebound/rebound . When I use the pyclbr.readmodule_ex API:

source_code_data = pyclbr.readmodule_ex(source_code_module, path=source_code_path)

I specify it the module and it's path:

DEBUG:root:Source code module: __main__, Source code path: ['/tmp/rebound/rebound/rebound']

I then get this error:

File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/importlib/util.py", line 69, in _find_spec_from_path
    raise ValueError('{}.__spec__ is None'.format(name))
ValueError: __main__.__spec__ is None

I then tried to use the function that is not supposed to be used by the public: _readmodule :

source_code_data = pyclbr._readmodule(source_code_module, source_code_path, )

But I could not decide what should be the value of the parameter: inpackage .

Upon tracing the code via debugger, I spotted a mistake:

def _find_spec_from_path(name, path=None):
    """Return the spec for the specified module.

    First, sys.modules is checked to see if the module was already imported. If
    so, then sys.modules[name].__spec__ is returned. If that happens to be
    set to None, then ValueError is raised. If the module is not in
    sys.modules, then sys.meta_path is searched for a suitable spec with the
    value of 'path' given to the finders. None is returned if no spec could
    be found.

    Dotted names do not have their parent packages implicitly imported. You will
    most likely need to explicitly import all parent packages in the proper
    order for a submodule to get the correct spec.

    """
    if name not in sys.modules:
        return _find_spec(name, path)
    else:
        module = sys.modules[name]
        if module is None:
            return None
        try:
            spec = module.__spec__
        except AttributeError:
            raise ValueError('{}.__spec__ is not set'.format(name)) from None
        else:
            if spec is None:
                raise ValueError('{}.__spec__ is None'.format(name))
            return spec

This is the function in the module: python3.8/importlib/util.py and it evaluates __main__ as a built-in module as it falls in the else block.

How do I differentiate __main__ of my target source code to read from the built-in __main__ ? In other words, how do I read the module __main__ of the codebase: rebound?

Answer 1

TL:DR

Try:

source_code_data = pyclbr.readmodule_ex("rebound.__main__", path=source_code_path)

Explanation

As you already know: _find_spec_from_path will search for name in sys.modules and __main__ is always present there.

If you inspect sys.modules.keys() you'll notice that it contains dot separated module names. Example from Ipython shell:

 'IPython.display',
 'IPython.extensions',
 'IPython.extensions.storemagic',
 'IPython.lib',
 'IPython.lib.backgroundjobs',
 'IPython.lib.clipboard',
 'IPython.lib.display',
 'IPython.lib.pretty',
 'IPython.lib.security',
 'IPython.paths',

And if you realize you are looking for rebound.__main__ and not __main__ it becomes obvious. In order to step into if block the name can't be in sys.modules . The last remark would be that _find_spec_from_path has no bugs.

# python3.8/importlib/util.py
def _find_spec_from_path(name, path=None):
    # ...
    if name not in sys.modules:
        return _find_spec(name, path)
    else:
    #...