包或子包中所有功能的“ from src import *”动态导入

Question

Goal:
I want to be able to dynamically import all functions in a sub-package with 'a direct call'

Usage:
my project:

project/
|-- main.py
|-- src/
|---- __init__.py
|---- foo.py
|---- bar.py 

foo.py has just one function:

def foo_funct(): 
    print("foo")

bar.py has just one function:

def bar_funct():
    print("bar")

and finally main.py :

from src import * 
(...)
foo_funct()
bar_funct()
(...)

Comments:

1. if my __init__.py is something like this

    import os __all__ = [i.replace(".py", "") for i in os.listdir(os.getcwd()+"/src/") if "__" not in i] 

   I will be able to call foo.foo_funct() or bar.bar_funct() but not foo_funct() or bar_funct()

2. if my __init__.py is something like this:

    from src.foo import * from src.bar import * 

   I will be able to call foo_funct() or bar_funct() but for each new sub package I will have to modify my __init__.py

3. Assuming that from src import * is not the most pythonic method, and assuming that it could be very dangerous to have direct calls due to possible naming conflicts such as a.tree_funct() and b.tree_funct() , is there any method to reach my goal?

Answer 1

Personally, I prefer to keep things explicit, and just import names that are part of the package API into __init__ explicitly. Your project won't change so rapidly that dynamically importing everything into the __init__.py is going to be a time saver.

But if you want to do this, then you have a few options here. If you need to support Python versions older than 3.7, then you can update the package namespace by poking at the globals() dictionary . List all .py files and import them using importlib.import_module() (or with __import__() if you need to support Python versions before 2.7):

__all__ = []

def _load_all_submodules():
    from pathlib import Path
    from importlib import import_module

    g = globals()
    package_path = Path(__file__).resolve().parent
    for pyfile in package_path.glob('*.py'):
        module_name = pyfile.stem
        if module_name == '__init__':
            continue
        module = import_module(f'.{module_name}', __package__)
        names = getattr(
            module, '__all__', 
            (n for n in dir(module) if n[:1] != '_'))
        for name in names:
            g[name] = getattr(module, name)
            __all__.append(name)

_load_all_submodules()
del _load_all_submodules

The above keeps the namespace clean; after the _load_all_submodules() function has run it is removed from the package. It uses the __file__ global to determine the current path and finds any sibling .py files from there.

If you only need to support Python 3.7 and up, you can define module-level __getattr__() and __dir__() functions to implement dynamic lookups.

Using those hooks in your package __init__.py file could look like:

def _find_submodules():
    from pathlib import Path
    from importlib import import_module

    package_path = Path(__file__).resolve().parent
    return tuple(p.stem for p in package_path.glob('*.py') if p.stem != '__init__')

__submodules__ = _find_submodules()
del _find_submodules


def __dir__():
    from importlib import import_module
    names = []
    for module_name in __submodules__:
        module = import_module(f'.{module_name}', __package__)
        try:
            names += module.__all__
        except AttributeError:
            names += (n for n in dir(module) if n[:1] != '_')
    return sorted(names)


__all__ = __dir__()


def __getattr__(name):
    from importlib import import_module
    for module_name in __submodules__:
        module = import_module(f'.{module_name}', __package__)
        try:
            # cache the attribute so future imports don't call __getattr__ again
            obj = getattr(module, name)
            globals()[name] = obj
            return obj
        except AttributeError:
            pass
    raise AttributeError(name)