pip Virtualenv vs. directory in project

Question

Why does pip need to use virtual environments to isolate packages per-project, instead of just installing them in a default directory in the project? This seems like added complexity without benefit.

NPM, for example, installs packages in the <project_root>\\node_modules by default. No virtual environment necessary, and packages are still installed in a project-independent way.

Edit: To be clear, I'm interested in the practical advantages to pip's use of virtual environments over package management systems like NPM, Nuget, and Webpack, which all use directories in the project. Otherwise, if this is just a limitation of Python's modules system, then I'd be interested to know that too.

Answer 1

Because Python's module system doesn't work that way. If pip were to install, say, requests by just downloading it to a python_modules directory, that wouldn't be enough to for import requests to work; it would have to be import python_modules.requests , but then we'd still have problems whenever requests tried to import one of its dependencies, as that would need python_modules prepended, too, and it'd just be a big mess. The solution that virtual environments use is to modify the PYTHONPATH environment variable to include python_modules , plus some extra stuff to take care of executable scripts and not importing packages from outside the virtualenv.

Answer 2

I think maybe you don't know what a virtual environment actually is.

If you were to put some module in a project-specific directory, like myproj/modules , then you would have to add myproj/modules to the search path that Python uses so that you module can be found. One way to do that is to define or modify the environment variable PYTHONPATH . Any directories listed in that variable will be searched for modules, in addition to some hard-coded set of directories.

$ export PYTHONPATH=./myproj/modules

However, that's really all a virtual environment is. The directory contains the desired version of Python, along with whatever modules you want to use. The activate script you run to "enable" a virtual environment does little more than set the value of PATH and PYTHONPATH so that anytime you run python , both the correct version is used and your project-specific set of modules is used in place of any global library.