Installing local packages with Python virtualenv --system-site-packages

Question

I know this topic has been beat to death but I have not been able to find a solution to the problem I'm having on SO or elsewhere, so I suspect that there may be a bug somewhere in my system.

I am on an older RHEL 6 platform with Python 3.4. I am developing an application that will run on this platform that uses Qt. I've installed all of the relevant libraries via yum (eg qt-devel , pyqt4-devel , etc.) and now want to install my application package as an "editable" package using pip install -e mypkg . I also have a couple of dependency requirements that are not on yum and must be installed via pip .

What I would like to do is create a virtualenv that "inherits" the system packages installed via yum but allows me to pip install my own packages into a virtualenv directory in my home directory.

From my Googling it looks like the best way to do this is to create a virtual env with the system's site packages directory:

$ python3 -m venv --system-site-packages ~/venv

However, when I try to install a package to this virtualenv's site-packages directory, it attempts to install it under /usr/lib and I get a Permission denied error.

So it appears that the --system-site-packages option makes my virtualenv completely share the site-packages directory from my system instead of using it as a "base", where further packages can be layered on top.

This answer states that using pip install -I should do what I want, but that does not appear to be the case:

(venv) $ pip3 install -I bitstring
...
error: could not create '/usr/lib/python3.4/site-packages/bitstring.py': Permission denied

Answer 1

Create the virtual environment without the --system-site-packages switch. After the environment was created go to the folder the environment was created in. It should have a file pyvenv.cfg . Edit this file. It has (among other text) a line

include-system-site-packages = false

Change this line to:

include-system-site-packages = true

Activate the environment. Module installations will now go to the virtual environment and the system site packages are visible too.

Answer 2

With Python 3.8, it seems --system-site-packages work as expected:

python3 -m venv --system-site-packages myProject
cat myProject/pyvenv.cfg 

home = /usr/bin
include-system-site-packages = true
version = 3.8.5

After installation astroid, isort, wrapt , I got:

pip list -v
Package                Version              Location                                                Installer
---------------------- -------------------- ------------------------------------------------------- ---------
apturl                 0.5.2                /usr/lib/python3/dist-packages                                   
astroid                2.4.2                /home/to/no/MR/auto-gen/lib/python3.8/site-packages pip      
isort                  5.6.4                /home/to/no/MR/auto-gen/lib/python3.8/site-packages pip      
jedi                   0.15.2               /usr/lib/python3/dist-packages                                   
keyring                18.0.1               /usr/lib/python3/dist-packages                                   
wrapt                  1.12.1               /home/to/no/MR/auto-gen/lib/python3.8/site-packages pip

Already installed 'system' packages are taken from /usr/lib/python3/dist-packages while locally (venv) installed packages from: /home/to/no/MR/auto-gen/lib/python3.8/site-packages