Pycharm not recognizing project packages after copying from old version of project

Question

I'm trying to set up a unit testing framework for a repo using pytest, and I was working in an old branch. I cloned the new branch and copied the files over to the new branch. When invoking pytest, the tests would run fine, but they were referencing files in the old project (even though they were imported as a package in the current project and not through a direct file path).

This is the line/command used to run pytest:

(venv) C:\\Program Files\\Python-3.8.2\\pycharm_projects\\fnma-erm01-devl-edl-hao-repo_21.3>python -m pytest

To attempt to remedy that issue, I deleted the old branch locally, and now none of the tests will work, and it doesn't recognize any of the project directories as valid modules.

Stacktrace:

========================================================================================================================================================================= test session starts =========================================================================================================================================================================
platform win32 -- Python 3.8.2, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: C:\Program Files\Python-3.8.2\pycharm_projects\ERM_EDL_HAO_21.3_Branch, configfile: pytest.ini
plugins: mock-3.5.1
collecting ... sys path ['
C:\\Program Files\\Python-3.8.2\\pycharm_projects\\ERM_EDL_HAO_21.3_Branch\\SourceCode\\application\\deploy\\app\\hao',
'C:\\Program Files\\Python-3.8.2\\pycharm_projects\\ERM_EDL_HAO_21.3_Branch',
'C:\\Progra~1\\Python-3.8.2\\pycharm_projects\\fnma-erm01-devl-edl-hao-repo\\SourceCode\\application\\deploy\\app\\hao\\lambda',
'C:\\Progra~1\\Python-3.8.2\\pycharm_projects\\fnma-erm01-devl-edl-hao-repo\\SourceCode\\application\\deploy\\app\\hao\\lambda\\tests',
'C:\\Progra~1\\Python-3.8.2\\pycharm_projects\\fnma-erm01-devl-edl-hao-repo\\SourceCode\\application\\deploy\\app\\hao\\lambda\\utils',
'C:\\Program Files\\Python-3.8.2\\pycharm_projects\\ERM_EDL_HAO_21.3_Branch',
'C:\\Program Files\\Python-3.8.2\\python38.zip',
'C:\\Program Files\\Python-3.8.2\\DLLs',
'C:\\Program Files\\Python-3.8.2\\lib',
'C:\\Program Files\\Python-3.8.2',
'C:\\Program Files\\Python-3.8.2\\pycharm_projects\\ERM_EDL_HAO_21.3_Branch\\venv',
'C:\\Program Files\\Python-3.8.2\\pycharm_projects\\ERM_EDL_HAO_21.3_Branch\\venv\\lib\\site-packages',
'C:\\Users\\g3uj2m\\AppData\\Roaming\\Python\\Python38\\site-packages',
'C:\\Users\\g3uj2m\\AppData\\Roaming\\Python\\Python38\\site-packages\\win32',
'C:\\Users\\g3uj2m\\AppData\\Roaming\\Python\\Python38\\site-packages\\win32\\lib',
'C:\\Users\\g3uj2m\\AppData\\Roaming\\Python\\Python38\\site-packages\\Pythonwin',
'C:\\Program Files\\Python-3.8.2\\lib\\site-packages']
collected 0 items / 16 errors  
=============================================================================================================================================================================== ERRORS ================================================================================================================================================================================
_______________________________________________________________________________________________________________________ ERROR collecting SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_amtm_upstream_notification.py _______________________________________________________________________________________________________________________
ImportError while importing test module 'C:\Program Files\Python-3.8.2\pycharm_projects\ERM_EDL_HAO_21.3_Branch\SourceCode\application\deploy\app\hao\lambda\tests\handlers\unittest_hao_amtm_upstream_notification.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
..\..\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
SourceCode\application\deploy\app\hao\lambda\tests\handlers\unittest_hao_amtm_upstream_notification.py:8: in <module>
    from hao_amtm_upstream_notification import lambda_function
E   ModuleNotFoundError: No module named 'hao_amtm_upstream_notification'

# ... ^ that part more or less repeated 15 more times ...

======================================================================================================================================================================= short test summary info =======================================================================================================================================================================
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_amtm_upstream_notification.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_batchjob_status_check.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_batchjob_submit.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_capture_hedge_status.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_ccfa_upstream_notification.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_dynamodb_status_check.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_glue_function.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_irdb_upstream_notification.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_krd_upstream_notification.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_ln_pool_max.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_s3_data_availability_check.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_stepfn_execution_status.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_swap_upstream_notification.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_trigger_sns_msg.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_verify_upstream_data_availability.py
ERROR SourceCode/application/deploy/app/hao/lambda/tests/handlers/unittest_hao_workflow_trigger.py

File tree:

pytest.ini

SOURCECODE
├───application
│   └───deploy
│       └───app
│           └───hao
│               ├───lambda
│               │   ├───hao_amtm_upstream_notification
│               │   │       lambda_function.py
│               │   │
│               │   ├───hao_batchjob_status_check
│               │   │       lambda_function.py
│               │   │
│               │   ├───hao_batchjob_submit
│               │   │       lambda_function.py
│               │   │
                  ... etc ...
│               │   │
│               │   ├───tests
│               │   │   conftest.py
│               │   │
│               │   │   ├───handlers
│               │   │   │       unittest_hao_amtm_upstream_notification.py
│               │   │   │       unittest_hao_batchjob_status_check.py
│               │   │   │       unittest_hao_batchjob_submit.py
│               │   │   │     ... etc ...

pytest.ini file:

# pytest.ini
[pytest]
python_files = unittest_*.py

Here's what I've already tried/checked:

1. All of the necessary folders are already marked as "Sources Root".
2. I've added __init__.py to all of the necessary folders.
3. I've checked to make sure __init__.py is not recognized as a text file.
4. I've run the "Invalidate Caches / Restart..." option under File.
5. I've ensured that the project's interpreter is aligned with the current project and directory.

It won't let me include screenshots, but the project structure should not be an issue as it worked perfectly fine in the old branch. (The old branch structure is identical to the new one, so there should be no reference issues when copying from one to the other)

EDIT 1: I'm using the pytest.ini configuration file, and the only thing specified in it is the custom prefix to identify test files. I created a new venv for the new project, and I'm invoking pytest from the built-in terminal in PyCharm, so the venv is active.

EDIT 2: Shortened the stack trace so it's easier to read, and adding the relevant parts of the file tree and the pytest.ini file.

Answer 1

Okay, it seems that Pycharm wasn't able to detect a change in the source roots after copying to a new project because, other than the project name, the paths are identical. Strangely enough, if I create a test configuration and run through Pycharm's built-in pytest runner, it works. That being said, I haven't been able to get the console to use the correct PYTHONPATH, but I found a workaround. Instead of invoking pytest through the console with python -m pytest , I created a script that forces PYTHONPATH to update.

New command:

python run_unittests.py

New script:

import pytest, os, sys

project_dir = os.path.dirname(os.path.abspath(__file__))
base_test_dir = os.path.join(project_dir, 'SourceCode', 'application', 'deploy', 'app', 'hao', 'lambda')

test_dirs = [
    os.path.join(base_test_dir, 'tests'),
    os.path.join(base_test_dir, 'handlers'),
    os.path.join(base_test_dir, 'utils'),
    base_test_dir
]

for i in test_dirs:
    if i not in sys.path:
        sys.path.append(i)

test_args = [a for a in sys.argv if a != 'run_unittests.py']

pytest.main([base_test_dir]+test_args)

Thanks @bad_coder for the help.