How does pip install know what dependancies to install?

Question

I'm confused about how pip knows what dependancies to install for a package.

First, I created a new virtual environment

% python3 -m venv env

Activate it

% source env/bin/activate

Listed installed packages. And I just see the default installed.

% pip list
Package    Version
---------- -------
pip        19.2.3
setuptools 41.2.0

Then, I installed another package (requests)

% pip install requests

And when I list packages installed again, I see a lot more has been installed than I asked. I get that it's installing these because pip knows that I need them for the package, but how?

Package    Version
---------- ---------
certifi    2020.6.20
chardet    3.0.4
idna       2.10
pip        19.2.3
requests   2.24.0
setuptools 41.2.0
urllib3    1.25.10

I was thinking there must be a requirements.txt in the root of the package, but I don't see it.

% cd env/lib/python3.8/site-packages/requests
% ls | grep requests.txt | wc -l
       0

So how does pip know? Does it look at every file and install all packages that are imported or is there some other magic going on here?

Answer 1

When you upload a package to PyPI, you must also upload other files like setup.py, setup.cfg, LICENSE.txt. In the file setup.py you have to call the setup() method. There you can specify the install_requires atriubut with an array of strings. In this array you can, as the name says, specify the installation requirements.

You can find further information under: How to upload your python package to PyPi

Conclusion: I don't think there will be a file in the package itself. It's more likely that a request will be sent to PyPI and the server will respond with the packages to install.

Answer 2

Projects are packaged and published as distribution files. Those files (archives) contain not only the Python code but also some metadata . One of the most important values in such metadata is Requires-Dist , which (in short and simplified) lists all other projects it depends on (ie pip should install those as well).

Developers of Python projects have to explicitly declare these dependencies. Depending on the packaging tool used for the project there are different ways to specify those dependencies:

• setuptools
• poetry
• flit
• distutils