sys.path() and PYTHONPATH issues

Question

I've been learning Python, I'm working in 2.7.3, and I'm trying to understand import statements.

1. The documentation says that when you attempt to import a module, the interpreter will first search for one of the built-in modules.

   What is meant by a built-in module?

2. Then, the documentation says that the interpreter searches in the directories listed by sys.path, and that sys.path is initialized from these sources:

   • the directory containing the input script (or the current directory).
   • PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH ).
   • the installation-dependent default.

   Here is a sample output of a sys.path command from my computer using python in command-line mode: (I deleted a few so that it wouldn't be huge)

    ['', '/usr/lib/python2.7', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntuone-couch', '/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol']

   Now, I'm assuming that the '' path refers to the directory containing the 'script', and so I figured the rest of them would be coming from my PYTHONPATH environmental variable. However, when I go to the terminal and type env , PYTHONPATH doesn't exist as an environmental variable. I also tried import os then os.environ , but I get the same output.

   Do I really not have a PYTHONPATH environmental variable? I don't believe I ever specifically defined a PYTHONPATH environmental variable, but I assumed that when I installed new packages they automatically altered that environment variable. If I don't have a PYTHONPATH, how is my sys.path getting populated? If I download new packages, how does Python know where to look for them if I don't have this PYTHONPATH variable?

3. How do environment variables work? From what I understand, environment variables are specific to the process for which they are set, however, if I open multiple terminal windows and run env , they all display a number of identical variables, for example, PATH . I know there file locations for persistent environment variables, for example /etc/environment , which contains my PATH variable. Is it possible to tell where a persistent environment variable is stored? What is the recommended location for storing new persistent environment variables? How do environment variables actually work with say, the Python interpreter? The Python interpreter looks for PYTHONPATH , but how does it work at the nitty-gritty level?

Answer 1

So many questions in one go! :)

Well I try to answer only a few of them.

1) a built-in module is any module that comes with a python release. For instance the sys and os modules are built-in modules. That's it really.

2) The PYTHONPATH variable don't exist by default on your system. When you launch the python interpreter, it fills the array of path where it search for modules, in the way you described. This is the result of sys.path. However sys.path is not the environment variable PYTHONPATH. If you set the PYTHONPATH in your system, then all the path contained in it will be included in the array that python's interpreter uses to search for modules.

I will leave the answer to the environment variables for others, as I don't feel I'm the right person to answer such a question. My feeling though, is that it might change from system to system. Anyway...

Hope it helps.

Answer 2

1. Built-in modules are the modules listed under sys.builtin_module_names . These modules are compiled together with the interpreter, and hence are always available. Note that they are part of the standard library but the standard library includes many more modules. To make the difference clear, if you try to run this code in python3.3:

    >>> import sys >>> sys.path = [] >>> import os # works >>> import traceback # ImportError, traceback is in the stdlib!

   On python2 both succeed because the import special-cases the standard library path. (See this for more information on the changes to import in python3.3).

2. If the PYTHONPATH isn't found then it doesn't include any extra directory. However the documentation you link also states that python uses an "installation-dependent default". The directories you are seeing in your sys.path are defined in this "installation-depedent default". If you define the PYTHONPATH you can add other directories to sys.path before the default one, but no harm is done without defining it.

3. Environment variables are defined per-process however shells can provide their own "scope" of variables to the subprocesses they launch. For example you can set environment variables in the shell files like ~/.bashrc or ~/.profile .

   If you are on Ubuntu the preferable way of defining a system wide and persisten environment variable is using the pam_enviroment . You can see this question on Ask Ubuntu that shows how you can set a system wide environment variable.

   AFAIK there is no standard way of setting them for every (eg) linux distro, and, obviously, each OS has its own method of defining them.

   If you want a more detailed explanation about environment variables handling you should probably ask on Super User .

Answer 3

PYTHONPATH is not defined on your system. That means all you see in sys.path except for '' is "the installation-dependent default."

Environment variables characterize the environment, not a process. However, you can tweak them when starting a process by using, say, Linux env command. That basically means that the process will run in a different environment. The "how if works" part of your question can have platform-dependent answers. However, I don't think you can "tell where a persistent environment variable is stored", if you mean a file. Environment variables can be set in any of the files that are executed at some point (usually at startup) or just in the command line.