The question of unix symbolic inspired from the usage of python venv executable symlink

Question

Question brief description

After creation of venv (either via python -m venv.venv/ or virtualenv.venv/ ), a files tree will be created. There is a file .venv/bin/python or python3 , python3.<x> . It is a unix symbolic link to /usr/bin/python . But how this symbolic link be launched with import search path for venv?

My effort

There is little resource for this problem. But I have made some experiment:

The sys.path values for those two entries:

For \usr\bin\python

>>> import sys
>>> sys.path
['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages']

For /path/to/venv/bin/python

>>> import sys
>>> sys.path
['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/path/to/venv/lib/python3.10/site-packages']

But when I create a directory /path/to/empty/ and run ln -s /usr/bin/python /path/to/empty/bin/python . The structure of /path/to/empty is like below:

$ tree .
.
├── bin
│   └── python -> /usr/bin/python
└── lib
    └── python3.10
        └── site-packages

Those sub empty directories are all manually created.

Then when I launch /path/to/empty/bin/python

>>> import sys
>>> sys.path
['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages']

There is no difference against /usr/bin/python and /path/to/empty/bin/python .

According to my knowledge to unix executable, without providing arguments, the information an executable can access is environment variable, current working directory (It can be regarded as a special environment variable) and nothing else.

In the experiments above, those executable entries are launched in exact same bash console, without any preset local environment variable. And it also seems not affected by current working directory (according to /path/to/empty/ ).

Question in general

It indicates that there is some hidden powerful feature of Unix symbolic links. Is it true?

For user aspect, what extra capability of symbolic link have which are beyond hard links.

If so, how to utilize those powerful features?

Answer 1

This is not the product of some hidden superpower in Unix symbolic links. When you run python -m venv.venv/ or virtualenv.venv/ , you should find that .venv/pyvenv.cfg has been created for you. When you then start up the python executable, it automatically imports the site module during initialization. This module automatically sets up your search paths. According to its documentation :

If a file named “pyvenv.cfg” exists one directory above sys.executable, sys.prefix and sys.exec_prefix are set to that directory and it is also checked for site-packages (sys.base_prefix and sys.base_exec_prefix will always be the “real” prefixes of the Python installation). If “pyvenv.cfg” (a bootstrap configuration file) contains the key “include-system-site-packages” set to anything other than “true” (case-insensitive), the system-level prefixes will not be searched for site-packages; otherwise they will.

Some testing to demonstrate:

/usr/bin/python -c import sys; print(sys.path) /usr/bin/python -c import sys; print(sys.path) results in ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages'] because no pyvenv.cfg file exists in the directory above the python executable.

.venv/bin/python -c 'import sys; print(sys.path)' .venv/bin/python -c 'import sys; print(sys.path)' results in ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/path/to/venv/lib/python3.10/site-packages'] because a pyvenv.cfg file was automatically created in the right spot and the site module found it.

.venv/bin/python -S -c 'import sys; print(sys.path)' .venv/bin/python -S -c 'import sys; print(sys.path)' results in ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages'] because using -S disables automatic importing of site , so it never looked for pyvenv.cfg .

If you delete the pyvenv.cfg file, you can see that the site module does not modify the search path like it did before: .venv/bin/python -c 'import sys; print(sys.path)' .venv/bin/python -c 'import sys; print(sys.path)' (without the -S ) results in ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages'] .