Install package dependencies with setup.py and wheels
Question
We're using an internally hosted PyPI server ( devpi-server ) so that we can host binary wheels of huge packages that take a long time to install from source like scipy, matplotlib, etc. Installing these packages with pip install scipy works perfectly and definitely uses the wheel we've created. However, working with any of our internally developed python packages that have a dependency on one of these packages and running python setup.py install|develop|test|whatever results in the following error:
No local packages or download links found for scipy
Traceback (most recent call last):
File "setup.py", line 136, in <module>
'develop': DevelopCommand
File "/usr/local/lib/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/dist.py", line 239, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/dist.py", line 263, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/pkg_resources.py", line 564, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/pkg_resources.py", line 802, in best_match
return self.obtain(req, installer) # try and download/install
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/pkg_resources.py", line 814, in obtain
return installer(requirement)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_egg
return cmd.easy_install(req)
File "/users/me/virtualenvs/devpi-test/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 587, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('scipy')
And with easy_install :
$ easy_install scipy
Searching for scipy
Reading http://pypi.internal.example.com/us/base/+simple/scipy/
No local packages or download links found for scipy
error: Could not find suitable distribution for Requirement.parse('scipy')
If I grab the URL it's looking at I get:
$ curl http://pypi.internal.example.com/us/base/+simple/scipy/
<html>
<head>
<title>us/base: links for scipy</title></head>
<body>
<h1>us/base: links for scipy</h1>
<form action="http://pypi.internal.example.com/us/base/+simple/scipy/refresh" method="post"><input name="refresh" type="submit" value="Refresh PyPI links"/></form>
us/external <a href="../../../external/+f/c48/5006bc28a8607/scipy-0.14.0-cp27-none-linux_x86_64.whl#md5=c485006bc28a8607b2fc1331df452dc1">scipy-0.14.0-cp27-none-linux_x86_64.whl</a><br/>
</body></html>
If I request the URL listed in that output, I get the wheel:
$ curl --silent 'http://pypi.internal.example.com/us/external/+f/c48/5006bc28a8607/scipy-0.14.0-cp27-none-linux_x86_64.whl#md5=c485006bc28a8607b2fc1331df452dc1' \
| file -
/dev/stdin: Zip archive data, at least v2.0 to extract
1 answers
solution1
1 ACCPTED 2018-05-18 20:07:50
First of all, the issue described in the question does not depend on using devpi and can be reproduced with any PyPI server, including the main repo at pypi.org . Example:
# setup.py
from setuptools import setup
setup(name='spam', install_requires=['vprof>0.37'])
(Instead of vprof , you can take any other package that does not ship anything besides wheels)
Test it:
$ pip install --upgrade "setuptools<38.2.0"
...
Successfully installed setuptools-38.1.0
$ python setup.py install
running install
running bdist_egg
running egg_info
writing spam.egg-info/PKG-INFO
...
Processing dependencies for spam==0.0.0
Searching for vprof>0.37
Reading https://pypi.python.org/simple/vprof/
No local packages or working download links found for vprof>0.37
error: Could not find suitable distribution for
Requirement.parse('vprof>0.37')
Bang. Things get worse when you declare a binary-only package as a build dependency:
setup(name='spam', setup_requires=['vprof>0.37'])
Now all of the build and packaging commands will also fail, being unable to download build deps.
The issue solely depends on setuptools version used. Since 26 Nov 2017 and version 38.2.0, setuptools supports fetching and installing wheel dependencies , so if you still experience this issue:
upgrade setuptools
Newer OS releases should already ship a recent version of setuptools . For example, Ubuntu 18.04 has setuptools==39.0.1 by default ( link ). If you still have the old setuptools installed, most of the time it will be managed by the system package manager, so you shouldn't update it via pip . You can user-install an additional copy of setuptools
$ pip install --user --upgrade "setuptools>=38.2.0"
or use virtual environments for that matter.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.