Add package path to python .pth file using environment variables

Question

Is it possible to use environment variables to specify a library path in python .pth files and if yes, how so?

Background : My python project (lives in /path/to/my/project) runs in a virtualenv, but needs to use an external (but local) package. Using virtualenv with --sytem-site-packages is not an option, therefore I added the path to the external package via a .pth file as suggested by this answer . The .pth lies in /path/to/my/project/lib/python2.7/site-packages and contains only the following line:

/path/to/external_dir/etc/python

This works as expected and allows me to import the external library.

In order to use my project on a different computer, I want to use an environment variable in the path instead of an absolute path. I can be sure that the environment variable EXTERNAL_DIR is set as follows before python is started:

export EXTERNAL_DIR="/path/to/external_dir"

But if /path/to/external_dir in the .pth file is replaced by ${EXTERNAL_DIR} I get an import error. The content of the .pth file then looks like this:

${EXTERNAL_DIR}/etc/python

Is this simply not the correct way to use the environment variable or is it not possible to use them in .pth files at all?

I'm using Debian, bash and python 2.7.

Answer 1

I don't believe that it's possible to refer to environment variables directly in .pth files as you propose. See the site documentation for the acceptable format for these files (Python 2 , 3 ).

It is possible to use an environment variable as you want, however, using an interesting property of .pth files mentioned there:

Lines starting with import (followed by space or tab) are executed.

This behavior can be leveraged in clever and possibly dangerous ways, to have arbitrary Python code execute as part of path construction (see Graham Dumpleton's excellent post here for more on this).

To support path configuration via environment variable, create a .pth file with this content:

import os, sys; dir = os.getenv('EXTERNAL_DIR'); dir and sys.path.append(dir)

This will work on both Python 2 and 3; the import trick only works if the code can be expressed in a single line.

This code will append the value of EXTERNAL_DIR to sys.path if the environment variable is defined, and do nothing if not.

# Create the .pth file.
$ echo "import os, sys; dir = os.getenv('EXTERNAL_DIR'); dir and sys.path.append(dir)" > /path/to/python/site-packages/extra.pth

# With no environment var, file does nothing:
$ python -c 'import sys; print(sys.path)[-1]'
/path/to/python/site-packages

# With environment variable, EXTERNAL_DIR gets added to end of path:
$ EXTERNAL_DIR=/foo/bar python -c 'import sys; print(sys.path)[-1]'
/foo/bar