Three-way Python import chain results in ModuleNotFoundError or ImportError

Question

Consider a project structure --

\root
    __init__.py
    bar.py
    \package1
        __init__.py
        abstractFoo.py
        implementedFoo.py
   

And a few file definitions for context --

• bar.py: project's primary functionality
• abstractFoo.py: defines Foo
• implementedFoo.py: implements Foo (numerous of these files exist under package1 corresponding to distinct implementations of Foo)

Lastly, the imports --

• bar.py imports helper functions from abstractFoo.py (ie from package1.abstractFoo import validateFoo, loadFoo ...); bar.py leverages said helper functions to import various implementedFoo.py on-demand
• implementedFoo.py imports abstract classes from abstractFoo.py (see below for attempts)
• abstractFoo.py imports nothing

The clash occurs in bar.py, when trying to import implementedFoo.py on-demand, specifically when implementedFoo.py itself tries importing from abstractFoo.py.

It seems both relative (eg implementedFoo.py: from . import abstractFoo) and absolute (eg implementedFoo.py: from root.package1 import abstractFoo) imports fail as they are no longer "correct" from the vantage point of bar.py at runtime.

Note I am running Python 3.10.x. Please let me know if I can provide any additional information. I am appending below a sample error...

In [2]: Source('bloomberg')
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-c290b4e239b8> in <cell line: 1>()
----> 1 Foo('Foo')

<ipython-input-1-a0ec8d9f6e48> in __init__(self, Foo_name)
     85     def __init__(self, name: str):
     86         self.Foo_name = Foo_name
---> 87         self.Foo = package1.abstractFoo.load_Foo(self._Foo_path(self.Foo_name))
     88
     89     def read(self, s: str):

path\to\root\package1\abstractFoo.py in load_Foo(Foo_path)
     33     foo = importlib.util.module_from_spec(spec)
     34     sys.modules['foo'] = foo
---> 35     spec.loader.exec_module(foo)
     36     return foo

C:\Python310\lib\importlib\_bootstrap_external.py in exec_module(self, module)

C:\Python310\lib\importlib\_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

path\to\root\package1\implementedFoo.py in <module>
----> 3 import abstractFoo
      4 from typing import Optional, Union
      5 from datetime import date, datetime

ModuleNotFoundError: No module named 'abstractFoo'

Thank you in advance.

Answer 1

Creating a subpackage directory subpackage1 under package1,and placing abstractFoo in subpackage1 allows me to accomplish what I'm looking for via absolute imports. I would probably have been content with this solution if I'd come across it before posting this question, but still open for any enhancements/alternatives.

For some additional context on why the structure is so convoluted: root is a database folder, bar is the database driver, abstractFoo abstracts as much as possible from pulling data via third party sources, and implementedFoo files are created on a 1:1 basis against third parties, each implementing common functions and returning data in a common format for database consumption. The database driver (bar) and API implementations (implementedFoo) must both import abstract classes and/or helper functions from abstractFoo. Upstream (ie beyond the root folder), only bar.py (or the root folder itself, TBD) will be imported to handle database i/o back-end.

Answer 2

You could use my new, experimental import library ultraimport and rather use relative imports

In implementedFoo.py you would then write:

import ultraimport
abstractFoo = ultraimport('__dir__/abstractFoo.py')

This import will always work, no matter how you run your script or what is your current working directory. You also don't need to create a new directory.