Importing project files in python

Question

I mostly worked on PHP where it was really easy to import classes specially after composer. But I am having hard times applying same directory structure in Python project.

I have the following directory structure.

backend commands indices.py backup.py etc.py components mapping product.py storage Es.py interfaces StorageInterface.py IndexInterface.py

My commands folder is basically the files i will run through cli where as the components will hold the actual business logic and interfaces just contains the ABCs for components.

I have to do several imports throughout the code some times from same folder sometimes from subfolder and sometimes same level another folder.

For example in my indices.py file i have this import

from components.storage.Es import Es

This gives me

root@07fc62bd3c97:/backend/commands# python indices.py --help 
Traceback (most recent call last):   
File "indices.py", line 1, in <module> from components.storage.Es import Es ImportError: No module named components.storage.Es

I have gone through other answers on SO and found really dirty solutions of putting sys and os modules and adding files in runtime. This could be okay when its 1 2 files but managing the whole project looks really bad.

My questions here are

1 - Am i following the pythonic way of managing a project 2 - How can i access all my classes files in a neat way

Thanks

Answer 1

You need __init__.py files in the commands , components , mapping , storage , and interfaces subdirectories in order for those directories and the files contained within them to be considered modules by python. Also, having individual modules which have the same name as some class which is defined inside of them is not really pythonic. For example, it's better to have an interfaces module defined by an interfaces.py file which contains both interface classes than to have two files named StorageInterface.py and IndexInterface.py which each contain a class definiton (I'm assuming that's what you're doing there).

Also, it matters what directory you start the python process from. In order for that import path to be meaningful, I'm pretty sure you'd need to start the python process from within the backend directory. Or you'd need to add the filesystem paths of the components , commands , etc. subdirectories to python's module path via the PYTHONPATH env var or by some other method.

Answer 2

Treat your entire backend folder like a package. If you're using legacy Python (Python 2), you'll need to put __init__.py files in all the folders so they are importable as subpackages. You'd need to either tweak your imports to then be from backend.components.storage.Es , or from ..components.storage.Es (relative imports).

Then, because it's all a package, you'll find that you can't go into a subfolder and execute code. The fix then is to run the code (eg backend/commands/backup.py ) by using the -m module flag: python -m backend.commands.backup from the folder containing backend . If you want, you can add an entrypoint later by making a setup.py that would turn it into a prettier command.