As part of the compilation step for a new python version, I fetch and run get-pip.py
, to have the latest pip installed next to the python executable:
$ /opt/python/3.7.0/bin/python --version
Python 3.7.0
$ /opt/python/3.7.0/bin/pip --version
pip 18.0 from /opt/python/3.7.0/lib/python3.7/site-packages/pip (python 3.7)
I have 25 such versions under /opt/python
, although I mostly use the five latest versions of each major.minor version that is not EOL. To setup an invironment I used to run virtualenv
or my virtualenvutils
with the -p /opt/python/XYZ/bin/python
option to get a virtual environment with a specific version.
With Python 3.7 this gives the imp module deprecation warning:
$ virtualenv -p /opt/python/3.7.0/bin/python /tmp/py37virtualenv
Running virtualenv with interpreter /opt/python/3.7.0/bin/python
Using base prefix '/opt/python/3.7.0'
/opt/util/virtualenvutils/lib/python3.6/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
New python executable in /tmp/py37virtualenv/bin/python
Installing setuptools, pip, wheel...done.
I have little hope this will be solved in virtualenv, as this has had a PendingDeprecationWarning
at least since 2014 (as can be seen from the output in this question )
While investigating replacing virtualenv
with python -m venv
in virtualenvutils
, I first created a new venv
based virtual environment by hand:
$ /opt/python/3.7.0/bin/python -m venv /tmp/py37venv
$ /tmp/py37venv/bin/pip --version
pip 10.0.1 from /tmp/py37venv/lib/python3.7/site-packages/pip (python 3.7)
That has an old pip
version! If you use it, you'll get:
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command
In the virtual environment created with virtualenv
you immediately get the latest version:
$ /tmp/py37virtualenv/bin/pip --version
pip 18.0 from /tmp/py37virtualenv/lib/python3.7/site-packages/pip (python 3.7)
I can run a post-creation step:
/tmp/py37venv/bin/pip install -U --disable-pip-version-check pip
which will take extra time. And if there was a some security update for pip
, this would imply running the non-secure version to get a secure version, an ideal point of attack.
From virtualenvutils
it is trivial to do the multiple steps to create a pip
-less virtualenv and then add pip
using get-pip.py
. From the command-line this is not so simple:
$ /opt/python/3.7.0/bin/python -m venv --without-pip /tmp/py37venvnopip
$ /tmp/py37venvnopip/bin/python -c "from urllib.request import urlopen; response = urlopen('https://bootstrap.pypa.io/get-pip'); open('/tmp/tmp_get_pip.py', 'w').write(response.read())"
$ /opt/python/3.7.0/bin/python /tmp/tmp_get_pip.py
......
$ /opt/python/3.7.0/bin/pip --version
pip 18.0 from /opt/python/3.7.0/lib/python3.7/site-packages/pip (python 3.7)
What is causing /opt/python/3.7.0/bin/python -m venv
to take that old pip
version? Is that the version available when 3.7.0 was released?
How can I update my install under /opt/python/3.7.0
in some way so that using /opt/python/3.7.0/bin/python -m venv
creates a virtualenv with the latest pip
version without reverting to scripts, aliases or using multiple commands? Having the latest pip
installed under /opt/python/3.7.0
obviously is not enough.
There are two bundled wheels:
/opt/python/3.7.0/lib/python3.7/ensurepip/_bundled/setuptools-39.0.1-py2.py3-none-any.whl
/opt/python/3.7.0/lib/python3.7/ensurepip/_bundled/pip-10.0.1-py2.py3-none-any.whl
I suspect I need to update those. Is there a better way than updating those by hand? Some option for /some/python -m venv
would be nice.
(And running /some/python -m ensurepip --upgrade
doesn't do the trick)
Running the deprecated /opt/python/3.7.0/bin/pyvenv
has the same old pip
version problem.
The trick is not to install the bundled version of pip (which will almost always be out of date), but to use it to install the most current version from the internet.
Standard library venv
offers a --without-pip
flag that can help here. After creating the virtual environment without pip, you can then you can "execute" ensurepip's wheel directly thanks to Python's zip importer. This is both faster and less hacky than installing pip and then immediately using that same pip installation to uninstall itself and upgrade.
Code speaks louder than words, so here's an example bash function for the process I've described:
# in ~/.bashrc or wherever
function ve() {
local py="python3"
if [ ! -d ./.venv ]; then
echo "creating venv..."
if ! $py -m venv .venv --prompt=$(basename $PWD) --without-pip; then
echo "ERROR: Problem creating venv" >&2
return 1
else
local whl=$($py -c "import pathlib, ensurepip; whl = list(pathlib.Path(ensurepip.__path__[0]).glob('_bundled/pip*.whl'))[0]; print(whl)")
echo "boostrapping pip using $whl"
.venv/bin/python $whl/pip install --upgrade pip setuptools wheel
source .venv/bin/activate
fi
else
source .venv/bin/activate
fi
}
If you prefer the older project virtualenv
, it also offers --no-pip
, --no-setuptools
, and --no-wheel
flags to achieve the same on Python 2.7.
Python 3.9's venv
may get an --upgrade-deps
option to automate this, see https://bugs.python.org/issue34556 for more info about that.
I use upgrade-ensurepip
to update those pip
and setuptools
wheel files that are part of the ensurepip
package. It's not as elegant as being able to upgrade ensurepip
via pip
, but it's still preferable to doing it manually.
It is an expected behavior. python -m venv
calls python -m ensurepip
to install pip
and This answer shows that ensurepip
would only install the bundled version even with --upgrade
option. There isn't any official option to update the bundled pip
and setuptools
.
Well I have also no good idea to fix this problem as it just is the designed behavior. I would like to give two suggestions:
Use pipenv
. It is really good! And it will be the next-generation official package manager in the future(Although there is a big problem related to current Pypi's structure. In short, a package manager can only decide the dependencies with downloading the whole package. This gives a huge difficulty to building dependencies graph.).
Implement your custom EnvBuilder
, actually there is an official example about this. And in the example, it also use get-pip.py
to install the latest pip
.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.