setup with submodules dependencies

Question

We have a python package which is also a git repo. It depends on other python packages, themselves git repos. We made the latter git submodules of the former. None of these are public, so no PyPI.

None of the other questions related to installing with submodule dependencies match our pattern. My question is not about finding (sub)packages with setuptools , nor is it about relative imports.

This is our structure:

package-repo/
    setup.py
    setup.cfg
    README.md
    .gitignore
    .gitmodules
    .git/
    submodule-repo/
        .git/
        .gitignore
        setup.py
        setup.cfg
        README.md
        submodule/
            __init__.py
            moduleX.py
            moduleY.py
    package/
        __init__.py
        moduleA.py
        moduleB.py
        subpackage1/

As is the case with requirements.txt, I naively though that something as follow would work out:

from setuptools import setup

setup(name='package',
      version='0.4.1',
      description='A package depending on other self made packages',
      url='git.ownnetwork.com',
      author='wli',
      author_email='wli@',
      license='Proprietary',
      packages=['package','package.subpackage1'],
      include_package_data=True,
      python_requires='>=3.7',
    classifiers=[
        'Natural Language :: English',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: 3.8',
        'Programming Language :: Python :: 3.9',
    ],
      install_requires=["SQLAlchemy",
                        "pandas",
                        "./submodule-repo"])

It doesn't work.

An alternative is to add the submodule in packages and indicate its directory in package_dir . Well it didn't work that well, and what's the point of making a setup.py in the "submodule" if it can't be installed when installing the dependent module? I just want it to be installed without having to put it on PyPI or create a virtual PyPI server, which is seriously overkill, or having to indicate what to do (ie pip install ./submodule-repo/ ) in the README.md, which is inelegant.

What's the way? Did I miss it in distutils or setuptools documentation?

Answer 1

You will need to specify where to install the submodule from.

install_requires=[
    'SQLAlchemy',
    'pandas',
     # Your private repository module
    '<dependency_name> @ git+ssh://git@github.com/<user_name>/<repo_name>@<branch>'
]