python distutils不包括SWIG生成的模块

Question

I am using distutils to create an rpm from my project. I have this directory tree:

project/
        my_module/
                 data/file.dat
                 my_module1.py
                 my_module2.py
        src/
            header1.h
            header2.h
            ext_module1.cpp
            ext_module2.cpp
            swig_module.i
        setup.py
        MANIFEST.in
        MANIFEST

my setup.py :

from distutils.core import setup, Extension

module1 = Extension('my_module._module',
                sources=['src/ext_module1.cpp',
                         'src/ext_module2.cpp',
                         'src/swig_module.i'],
                swig_opts=['-c++', '-py3'],
                include_dirs=[...],
                runtime_library_dirs=[...],
                libraries=[...],
                extra_compile_args=['-Wno-write-strings'])

setup(  name            = 'my_module',
        version         = '0.6',
        author          = 'microo8',
        author_email    = 'magyarvladimir@gmail.com',
        description     = '',
        license         = 'GPLv3',
        url             = '',
        platforms       = ['x86_64'],
        ext_modules     = [module1],
        packages        = ['my_module'],
        package_dir     = {'my_module': 'my_module'},
        package_data    = {'my_module': ['data/*.dat']} )

my MANIFEST.in file:

include src/header1.h
include src/header2.h

the MANIFEST file is automatically generated by python3 setup.py sdist . And when i run python3 setup.py bdist_rpm it compiles and creates correct rpm packages. But the problem is that when im running SWIG on a C++ source, it creates a module.py file that wraps the binary _module.cpython32-mu.so file, it is created with the module_wrap.cpp file, and it isnt copied to the my_module directory.

What I must write to the setup.py file to automatically copy the SWIG generated python modules?

And also I have another question: When I install the rpm package, I want that an executable will be created, in /usr/bin or so, to run the application (for example if the my_module/my_module1.py is the start script of the application then I can run in bash: $ my_module1 ).

Answer 1

The problem is that build_py (which copies python sources to the build directory) comes before build_ext , which runs SWIG.

You can easily subclass the build command and swap around the order, so build_ext produces module1.py before build_py tries to copy it.

from distutils.command.build import build

class CustomBuild(build):
    sub_commands = [
        ('build_ext', build.has_ext_modules), 
        ('build_py', build.has_pure_modules),
        ('build_clib', build.has_c_libraries), 
        ('build_scripts', build.has_scripts),
    ]

module1 = Extension('_module1', etc...)

setup(
    cmdclass={'build': CustomBuild},
    py_modules=['module1'],
    ext_modules=[module1]
)

However, there is one problem with this: If you are using setuptools, rather than just plain distutils, running python setup.py install won't run the custom build command. This is because the setuptools install command doesn't actually run the build command first, it runs egg_info, then install_lib, which runs build_py then build_ext directly.

So possibly a better solution is to subclass both the build and install command, and ensure build_ext gets run at the start of both.

from distutils.command.build import build
from setuptools.command.install import install

class CustomBuild(build):
    def run(self):
        self.run_command('build_ext')
        build.run(self)


class CustomInstall(install):
    def run(self):
        self.run_command('build_ext')
        self.do_egg_install()

setup(
    cmdclass={'build': CustomBuild, 'install': CustomInstall},
    py_modules=['module1'],
    ext_modules=[module1]
)

It doesn't look like you need to worry about build_ext getting run twice.

Answer 2

It's not a complete answer, because I don't have the complete solution. The reason why the module is not copied to the install directory is because it wasn't present when the setup process tried to copy it. The sequence of events is:

running install
running build
running build_py
file my_module.py (for module my_module) not found
file vcanmapper.py (for module vcanmapper) not found
running build_ext

If you run a second time python setup.py install it will do what you wanted in the first place. The official SWIG documentation for Python proposes you run first swig to generate the wrap file, and then run setup.py install to do the actual installation.

Answer 3

It looks like you have to add a py_modules option, eg:

setup(...,
  ext_modules=[Extension('_foo', ['foo.i'],
                         swig_opts=['-modern', '-I../include'])],
  py_modules=['foo'],
)

Using rpm to Install System Scripts in Linux , you'll have to modify your spec file. The %files section tells rpm where to put the files, which you can move or link to in %post , but such can be defined in setup.py using:

options = {'bdist_rpm':{'post_install':'post_install', 'post_uninstall':'post_uninstall'}},

Running Python scripts in Bash can be done with the usual first line as #!/usr/bin/python and executable bit on the file using chmod +x filename .