ModuleNotFoundError in Python project

Question

I have the following project structure in Python (the... means I have n crawler_.py files).

project
├── crawlers
│   ├── __init__.py
│   ├── crawler_1.py
│   ├── crawler_2.py
│   ...
│   ├── crawler_n.py
│   └── useful_functions.py
├── main.py
└── __init__.py

I need to import all crawlers from crawler into main, so I use this.

# main.py
from crawlers import crawler_1
from crawlers import crawler_2
...
from crawlers import crawler_n

But I also need useful_functions.py inside all crawler_.py files, so I use this in each one.

# crawler_.py
import useful_functions

But when I ran main.py I got ModuleNotFoundError: No module named 'useful_functions' when it tried to import crawler_1.

So I tried the following

# crawler_.py
from crawlers import useful_functions

And it works when I run main.py . The problem is that I might want run only one of the crawler_.py directly. Using this last import statement, I get ModuleNotFoundError: No module named 'crawlers' . Not sure how to address this problem, if there's something inside the code I should adjust or if the structure that I'm using is fundamentally wrong (I'm perfectly okay with adjusting the project structure).

Answer 1

You can use this inside the crawler_n.py

if __name__ == '__main__':
    import useful_functions
else:
    import crawlers.useful_functions as useful_functions

__name__ == '__main__' checks if the module is called or is imported and thus makes the imports accordingly.