relative paths for modules in python

Question

I've attempted a few different techniques trying to do something that to me seems doable but I guess I am missing some gotchas about python (using 2.7 but would like this to work also for 3.* if possible).

I am not sure about terminology like package or module, but to me the following seems quite a "simple" doable scenario.

This is the directory structure:

.
├── job
│   └── the_script.py
└── modules
    ├── __init__.py
    └── print_module.py

The content of the_script.py :

# this does not work
import importlib
print_module = importlib.import_module('.print_module', '..modules')

# this also does not work
from ..modules import print_module

print_module.do_stuff()

The content of print_module :

def do_stuff():
    print("This should be in stdout")

I would like to run all this "relative paths" stuff as:

/job$ python2 the_script.py

But the importlib.import_module gives various errors:

• if I just use 1 input parameter ..modules.print_module , then I get: TypeError("relative imports require the 'package' argument")
• if I use 2 input parameters (as in the example above), then I get: ValueError: Empty module name

On the other hand using the from ..modules syntax I get: ValueError: Attempted relative import in non-package .

I think the __init__.py empty file should be enough to qualify that code as "packages" (or modules? not sure about the terminology), but it seems there's something I am missing about how to manage relative paths.

I read that in the past people was hacking this using the path and other functions from import os and import sys , but according to the official docs (python 2.7 and 3.*) this should not be needed anymore.

What am I doing wrong and how could I achieve the result of printing the content modules/print_module.do_stuff calling it from a script in the "relative directory" job/ ?

Answer 1

If you follow the structure of this guide here: http://docs.python-guide.org/en/latest/writing/structure/#test-suite (highly recommend reading it all, it is very helpful) you will see this:

To give the individual tests import context, create a tests/context.py file:

 import os import sys sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) import sample 

Then, within the individual test modules, import the module like so:

 from .context import sample 

This will always work as expected, regardless of installation method.

Translated in your case this means:

root_folder
├── job
│   ├── context.py <- create this file
│   └── the_script.py
└── modules
    ├── __init__.py
    └── print_module.py

In the context.py file write the lines shown above, but import modules instead of import samples

Finally in your the_script.py : from .context import module and you will be set to go!

Good luck :)

Answer 2

If you are not sure about terminology go to very nice tutorials:

http://docs.python-guide.org/en/latest/writing/structure/#modules

and

http://docs.python-guide.org/en/latest/writing/structure/#packages

But for your structure:

.
├── job
│   └── the_script.py
└── modules
    ├── __init__.py
    └── print_module.py

just say in the the_script.py :

import sys
sys.append('..')
import modules.print_module

This will add parent directory to PYTHONPATH, and python will see directory 'parallel' to job directory and it will work.

I think that at the most basic level it is sufficent to know that:

1. package is any directory with __init__.py file
2. module is a file with .py , but when you are importing module you omit extension.

Answer 3

I found a solution using sys and os .

The script the_script.py should be:

import sys
import os
lib_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '../modules'))
sys.path.append(lib_path)

# commenting out the following shows the `modules` directory in the path
# print(sys.path)

import print_module

print_module.do_stuff()

Then I can run it via command line no matter where I am in the path eg:

• /job$ python2 the_script.py
• <...>/job$ python2 <...>/job/the_script.py