setup.py file using requirements.txt

Question

I've read a discussion where a suggestion was to use the requirements.txt inside the setup.py file to ensure the correct installation is available on multiple deployments without having to maintain both a requirements.txt and the list in setup.py .

However, when I'm trying to do an installation via pip install -e. , I get an error:

 Obtaining file:///Users/myuser/Documents/myproject Processing /home/ktietz/src/ci/alabaster_1611921544520/work ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/System/Volumes/Data/home/ktietz/src/ci/alabaster_1611921544520/work'

It looks like pip is trying to look for packages that are available on pip ( alabaster ) on my local machine. Why? What am I missing here? Why isn't pip looking for the required packages on the PyPi server?

Answer 1

I have done it before the other way around, maintaining the setup file and not the requirements file. For the requirements file, just save it as:

*

and for setup, do

from distutils.core import setup

from setuptools import find_packages

try:
    from Module.version import __version__
except ModuleNotFoundError:
    exec(open("Module/version.py").read())

setup(
    name="Package Name",
    version=__version__,
    packages=find_packages(),
    package_data={p: ["*"] for p in find_packages()},
    url="",
    license="",
    install_requires=[
        "numpy",
        "pandas"
    ],
    python_requires=">=3.8.0",
    author="First.Last",
    author_email="author@company.com",
    description="Description",
)

For reference, my version.py script looks like:

__build_number__ = "_LOCAL_"
__version__ = f"1.0.{__build_number__}"

Which Jenkins is replacing the build_number with a tag

Answer 2

This question consists of two separate questions, for the rather philosopihc choice of how to arrange setup requirements is actually unrelated to the installation error that you are experiencing.

First about the error: It looks like the project you are trying to install depends on another library ( alabaster ) of which you apparently also did an editable install using pip3 install -e. that points to this directory:

/home/ktietz/src/ci/alabaster_1611921544520/work

What the error tells you is that the directory where the install is supposed to be located does not exist anymore. You should only install your project itself in editable mode, but the dependencies should be installed into a classical system directory, ie without the option -e .

To clean up, I would suggest that you do the following:

# clean up references to the broken editable install
pip3 uninstall alabaster
# now do a proper non-editable install
pip3 install alabaster

Concerning the question how to arrange setup requirements, you should primarily use the install_requires and extras_require options of setuptools:

# either in setup.py
setuptools.setup(
    install_requires = [
        'dep1>=1.2',
        'dep2>=2.4.1',
    ]
)

# or in setup.cfg
[options]
install_requires =
    dep1>=1.2
    dep2>=2.4.1

[options.extras_require]
extra_deps_a = 
   dep3
   dep4>=4.2.3
extra_deps_b =
   dep5>=5.2.1

Optional requirements can be organised in groups. To include such an extra group with the install, you can do pip3 install.[extra_deps_name] .

If you wish to define specific dependency environments with exact versions (eg for Continuous Integration), you may use requirements.txt files in addition, but the general dependency and version constraint definitions should be done in setup.cfg or setup.py .