通过pip安装psutil（wheel）作为依赖项时遇到麻烦

Question

I wrote a package with a dependency's dependency to psutil ( my-package depends on third-party-package which depends on psutil ).
Since it's supposed to run on a server without any connectivity and without gcc , I prepared the deployment locally with a psutil python platform wheel and pip install my-package --download , then sent everything on the server.

Now everything is ready on the server, but for some reason, when I run the installation, pip refuses to install psutil . Note that the server is a red hat 7.2 running pip 7.1.0, virtualenv 1.10.1 and python 2.7.5 (and I can't change the version of anything).

$ pip install /tmp/python_packages/my-package-1.4.zip --no-index 
  --find-links /tmp/python_packages/ --use-wheel
Ignoring indexes: https://pypi.python.org/simple/

# blablabla, everything goes fine, then

Downloading/unpacking psutil (from third-party-package>=0.9->my-package==1.4)
  Could not find any downloads that satisfy the requirement psutil 
  (from third-party-package>=0.9->my-package==1.4)
Cleaning up...
No distributions at all found for psutil (from third-party-package>=0.9->my-package==1.4)
Storing complete log in /home/anto/.pip/pip.log

Here is what pip.log says:

Downloading/unpacking psutil (from third-party-package>=0.9->my-package==1.4)

  URLs to search for versions for psutil (from third-party-package>=0.9->my-package==1.4):
  # bla
  Skipping file:///tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl 
    because it is not compatible with this Python
  # bla
  Could not find any downloads that satisfy the requirement psutil (from third-
   party-package>=0.9->my-package==1.4)

Cleaning up...

So "not compatible with this Python", ok. But here is the really weird part: if I install psutil without anything else, and then the rest, everything goes fine.

$ pip install /tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl 
Unpacking /tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl
Installing collected packages: psutil
Successfully installed psutil
Cleaning up...
$ pip freeze -l
psutil==4.2.0
$ pip install /tmp/python_packages/my-package-1.4.zip --no-index 
  --find-links /tmp/python_packages/ --use-wheel

# blablabla

Successfully installed my-package third-party-package
Cleaning up...

What am I missing ? Any clue ?

Answer 1

The key issue you are facing is IMHO this:

Skipping file:///tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl 
because it is not compatible with this Python

A new recent feature has been introduced in pip to support "many linux" wheels (See this PEP )

When asking for a direct install of wheel, minimal checks are done and pip assumes you really want this installed.

When a wheel is installed indirectly as a dep of a dep in your case, there may be incompatible tags in this context.

I would ensure that I use the latest version of pip, setuptools and virtualenv to remove moving parts and it may fix you problem.

Vaguely related, and if this can help, I use this script with a Linux/Mac or Windows wrapper to perform consistent install of vendored pip packages. The key point is to vendor everything and this may be another reason why you get some failure.

Answer 2

Make sure that the version of wheel you use to create the wheel is the same as the version used during deployment.

I experienced the same problem when trying to deploy a wheel built with 0.29.0, using wheel 0.24.0.

I downgraded the version of wheel used to build the wheel to match the version used during deployment, and this resolved the problem.

Answer 3

From your answer, it seems that you got it working with the extra step.

When doing an offline install, you need to tell it where to find all the packages you want to install explicitly, not just the top level.

I personally have always been explicit about the fact you are looking in the file, and like any other pip install you can specify multiple installs on the same line for context.

$ pip install --no-index --find-links file:/tmp/python_packages/ --use-wheel psutil my-package-1.4.zip