Pip show dependency rationale for package installation during install

Question

When I am trying to install some packages with pip, sometimes some dependency fails to install, and I need to sort out why. A fundamental question is "why did I need to install package X at all?", but I cannot find any way to answer this from the pip install output, even with -vvv . Pip tells me what it is installing, but it doesn't say why it is doing it.

There are various tools to introspect dependency trees of packages, eg pipdeptree , especially when they are already installed, but these don't help me when the installation has failed. And it seems that internally pip must already have solved these dependencies and know why it has chosen to install a particular package. So how can I get it to share this information with me at install time?

Edit: It already shows this information when telling you what dependencies are already satisfied, eg

Requirement already satisfied: pillow>=6.2.0 in /data2/users/bfarmer/envs/bfarmer_dev_py  39/lib/python3.9/site-packages (from matplotlib>=1.3.1->stf-modelling) (8.0.1)
Requirement already satisfied: certifi>=2020.06.20 in /data2/users/bfarmer/envs/bfarmer_  dev_py39/lib/python3.9/site-packages (from matplotlib>=1.3.1->stf-modelling) (2020.12.5)

(at least I assume that it is what it is telling me at the end of those lines). But when I need this information the most, ie when something goes wrong, I get nothing:

Collecting PIL
  Downloading http://ehp.bom.gov.au/ehprepo/pypi/simple/pil/PIL-1.1.7.tar.gz (506 kB)
     |████████████████████████████████| 506 kB 8.1 MB/s
    ERROR: Command errored out with exit status 1:
    ...
    <traceback etc. follows>

In this example I am left wondering why the heck some package wanted PIL when pillow is already there. PIL is basically dead, so I need to update whatever package has a dependency on PIL to use pillow instead. But I have no idea what package that might be, and cannot figure out any way to find out. This seems like basic information, there must surely be a way to get it.

Answer 1

It kind of seems like no, pip cannot do this. I found this issue about it here:

https://github.com/pypa/pip/issues/53

It sounds like they are working on it, but nothing exists currently. I am still interested in workaround/third party solutions though, or advice from other developers about what they typically do in this situation. It kind of seems like I just have to manually trawl through all the dependencies of my dependencies, which just seems stupid. Maybe I can hack something in to the pip source...