Copy a non-Python file to specific directory during Pip Install

Question

Problem Statement: when I install my pip package, a specific file inside the package get coped to Temp directory

Approach:

My Package directory Sturcture is following:

my-app/
├─ app/
│  ├─ __init__.py
│  ├─ __main__.py
├─ folder-with-extra-stuff/
│  ├─ __init__.py
│  ├─ file_I_want_to_cppy.tar.gz 
├─ setup.py
├─ MANIFEST.in

I'm tweaking my setup.py file to do the job. Following is my setup.py

#!/usr/bin/env python

from setuptools import setup, find_packages
from setuptools.command.install import install
import os
import sys
import shutil

rootDir = os.path.abspath(os.path.dirname(__file__))

def run_custom_install():
 
    print("--------Start running custom command -------")
    temp_dir = r'c:\temp' if sys.platform == "win32" else r'/tmp'
    temp_col_dir = temp_dir + os.sep + 'dump'
    os.makedirs(temp_dir, exist_ok=True)
    os.makedirs(temp_col_dir, exist_ok=True)
    print("----------locate the zip file ---------------")
    ColDirTests = os.path.abspath(os.path.join(rootDir, 'my-app','folder-with-extra-stuff'))
    _src_file = os.path.join(ColDirTests , 'file_I_want_to_cppy.tar.gz ')
    print(f"******{_src_file}**********")
    if os.path.exists(_src_file):
        print(f"-----zip file has been located at {_src_file}")
        shutil.copy(_src_file, temp_col_dir)
    else:
        print("!!!!Couldn't locate the zip file for transfer!!!!")

class CustomInstall(install):
    def run(self):
        print("***********Custom run from install********")
        install.run(self)
        run_custom_install()

ver = "0.0.0"
setup(
    name='my_pkg',
    version=ver,
    packages=find_packages(),
    python_requires='>=3.6.0',
    install_requires = getRequirements(),
    include_package_data= True,
    cmdclass={
            'install' : CustomInstall,
            }
     )

MANIFEST.in

include README.md
include file_I_want_to_cppy.tar.gz
recursive-include my-app *
global-exclude *.pyc
include requirements.txt
prune test

Testing build:

> python setup.py bdist_wheel

It is working during build. I can see there is a directory formed C:\\temp\\dump and file_I_want_to_cppy.tar.gz inside it. But when I release the package in pip and try to install it from pip, the folder remains Empty!

Any idea what I might be doing wrong here?

Answer 1

After a lot of research I have figure out how to resolve this issue. Let me summarize my findings, it might be helpful for other who wants to do post_pip_install processing.

setup.py

• Different options to install package: 1) pip install pkg_name , 2) python -m setup.py sdist

• If you want to make them work in either ways, need to have install , egg_info and develop all 3 options repeated as shown in setup.py

• If you create *.whl file by python -m setup.py bdist_wheel , post pip install processing won't be executed! Please upload .tar.gz format generated using bdist to PyPi/Artifacts to make post pip install processing work. Again, Please note: It will not work when installing from a binary wheel

• upload the pip package: twine upload dist/*.tar.gz

   from setuptools import setup, find_packages from setuptools.command.install import install from setuptools.command.egg_info import egg_info from setuptools.command.develop import develop rootDir = os.path.abspath(os.path.dirname(__file__)) def run_post_processing(): print("--------Start running custom command -------") # One can Run any Post Processing here that will be executed post pip install class PostInstallCommand(install): def run(self): print("***********Custom run from install********") install.run(self) run_post_processing() class PostEggCommand(egg_info): def run(self): print("***********Custom run from Egg********") egg_info.run(self) run_post_processing() class PostDevelopCommand(develop): def run(self): print("***********Custom run from Develop********") develop.run(self) run_post_processing() ver = "0.0.0" setup( name='my_pkg', version=ver, packages=find_packages(), python_requires='>=3.6.0', install_requires = getRequirements(), include_package_data= True, cmdclass={ 'install' : PostInstallCommand, 'egg_info': PostEggCommand, 'develop': PostDevelopCommand } )

Few More Things from my research:

1. If you want to do pre-processing instead of post-processing , need to move install.run(self) at the end
2. while pip installing, if you want to see custom messages of pre/post instllation, use -vvv . Example: pip install -vvv my_pkg