Transitive import error: ModuleNotFoundError: No module named '…'

Question

I'm confused now. Here is the project tree:

project
- source
- - lib
- - - __init__.py
- - - utils.py
- - - stats.py
- test
- - lib
- - - test_stats.py

stats.py has import utils , which indeed works if one executes stats.py itself. Now test_stats.py has import lib.stats but that results in the ModuleNotFoundError: No module named 'utils' error if executed as PYTHONPATH=source pytest in the project directory:

==================================== ERRORS ====================================
___________________ ERROR collecting test/lib/test_stats.py ___________________
ImportError while importing test module '/lib/test_stats.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test/lib/test_stats.py:40: in <module>
    import lib.stats
source/lib/__init__.py:42: in <module>
    from .stats import Stats
source/lib/stats.py:40: in <module>
    import utils
E   ModuleNotFoundError: No module named 'utils'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.19 seconds ============================

What's going on and how to properly execute tests with such directory structure?

PS I'm aware of (development mode) pip install ... and tox tricks (for module search path), but I would like to understand whether they are absolutely necessary to get this going or I already assume something wrong in this simple setup.

Answer 1

import utils in Python 3 is absolute import, Python looks module utils in sys.path .

When you run stats.py as a script Python adds the directory project/source/lib/ to sys.path . So import utils in the script works.

But when you run test_stats.py Python doesn't add project/source/lib/ to sys.path . So import utils doesn't work.

A way to overcome that is to use relative import: from . import utils from . import utils . In stats.py it means: do not search sys.path , import module utils from the same directory as stats.py . But then you loose ability to run stats.py as a script. So move main code from stats.py to a separate script outside of lib/ .

A slightly different solution is to move main code from stats.py to module __main__.py and execute the module using python -m lib ( project/source/ must be in sys.path ).