Importing user-created Python modules and data files from relative directories

Question

I'm building a set of Python modules that depend on each other.

My file directory currently looks like:

.
├── utilities
│   ├── __init__.py
│   ├── utility.py
│   ├── data.csv
├── src
│   ├── __init__.py
│   |── functions
│       ├── __init__.py
│       └── function.py
└── __init__.py

Further, function.py imports data from utilities/data.csv .

From the top-level directory ( . ), I run python3 src/functions/function.py .

And I receive the following import error:

Traceback (most recent call last):
  File "src/functions/function.py", line 1, in <module>
    from utilities.utility import UtilityFunctionA
ImportError: No module named 'utilities'

How do I properly import utilities from the function.py file? Or should I not be running nested files in the first place, and instead running files at the top-level of the directory?

The imports are successful when running this code from within PyCharm.

Silly question, but I've been unable to figure it out despite reading a lot of documentation (and Googling).

---

UPDATE:

Using python3 -m src.functions.function works to run the module from the command line with proper imports and with successfully loading the csv.

However, when I then run the module from within PyCharm, for instance using

Now I receive the error that OSError: File b'utilities/data.csv' does not exist

Is there any way to setup my module to run both from within PyCharm and also from the command line?

Answer 1

If you want to be able to do - from utilities.utility import UtilityFunctionA - from within function.py , when running from the top level directory - . . You need to run the python file as a module , using -m option. Example -

python3 -m src.functions.function

---

But The above method only works if you always run from the . directory (top-level directory) . I always either add the top-level directory manually into the PYTHONPATH environment variable.

Or use os.path and relative path to add it to sys.path programmatically using __file__ . Example -

import sys
import os.path
path_to_top = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','..'))
sys.path.append(path_to_top)

After this do the import. If the directory structures would always remain the same, this would work even in other systems, without having to set any environment variable.

---

As per the updated requirements -

Now I receive the error that OSError: File b'utilities/data.csv' does not exist

Is there any way to setup my module to run both from within PyCharm and also from the command line?

For reading files, Python uses the current working directory as the start point , and all relative paths are resolved relative to the current working directory. It is never a good idea to rely on the current working directory to be a particular directory as we can run python scripts from anywhere using absolute path to the script. For loading files as well, we can use os.path and relative paths to create the absolute path to the file. Example -

import os.path
path_to_datacsv = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','..','utilities,'data.csv'))

Answer 2

This imports properly:

python3 -m src.functions.function

Based on How to do relative imports in Python?