如何透明地将绝对导入转换为相对内部外部模块？

Question

I have a problem: I need to use some external code as part of my own project. This seems to be a relatively common use case in machine learning research when trying to build new experiments on top of previously published solutions/models.

The file structure is as follows:

project_root/
├── external/
│   └── SomeoneElsesCode/
│       └── src/
│           └── dir1/
│               └── subdir1/
│                   ├── codeineed.py
│                   └── anciliarycode.py
└── src/
    └── MyModule/
        └── mycode.py

When trying to run the line

from external.SomeoneElsesCode.src.dir1.subdir1.codeineed import NeededClass

in mycode.py I am running into the problem with line

from src.dir1.subdir1.anciliarycode import AncilliaryClass

in codeineed.py since the external code is using absolute import paths. Since I do not control the code in SomeoneElsesCode I cannot simply adjust all import paths there. Is there any way to tell the Python interpreter to "relativize" all paths below SomeoneElsesCode ? If not, is there any recommended way of dealing with including external code in Python projects?

Answer 1

I believe that I have found an answer, though I hope that someone can suggest a better approach.

The solution to the problem seems to be to replace the builtin __import__() function with a wrapper preprocessing the path. This can be done by placing the following __init__.py file inside the external directory:

import builtins

default_import = builtins.__import__


class ExternalImport:
    def __init__(self, default_import) -> None:
        self.default_import = default_import

    def __call__(self, name, globals=None, locals=None, fromlist=(), level=0):
        if (
            ("src" in name)
            and (globals is not None)
            and (hasattr(globals, "__getitem__"))
            and (globals.get("__package__", None) is not None)
            and ("external" in globals.get("__package__", ""))
        ):
            package = globals["__package__"]
            externalLib = package.replace("external.", "").split(".")[0]
            name = f"external.{externalLib}.{name}"
        return self.default_import(name, globals, locals, fromlist, level)


builtins.__import__ = ExternalImport(default_import)

Essentially, I filter all import statements and look for those containing src and coming from the external module to adjust their names. This is not ideal since it requires all external packages to store their code in the src directory; however, I have not found any better way of detecting imports of actual packages (both built-in and installed).