Create Executable from a Python module with Pyinstaller or Cython

Question

I want to convert to an executable this python 2.7 project that has a module structure:

(.venv) ip-192-168-22-127:indictrans loretoparisi$ tree -L 1
.
├── __init__.py
├── __init__.pyc
├── __init__.spec
├── _decode
├── _utils
├── base.py
├── build
├── mappings
├── models
├── script_transliterate.py
├── tests
├── transliterator.py
└── trunk

I'm using pyinstaller for this at first stage I'm just doing:

pyinstall --onefile __init__.py

and I get a executable built:

192 INFO: PyInstaller: 3.3.1
192 INFO: Python: 2.7.10
201 INFO: Platform: Darwin-17.7.0-x86_64-i386-64bit
202 INFO: wrote /Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans/__init__.spec
208 INFO: UPX is not available.
209 INFO: Extending PYTHONPATH with paths
['/Users/loretoparisi/Documents/Projects/AI/indic-trans',
 '/Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans']
210 INFO: checking Analysis
218 INFO: checking PYZ
223 INFO: checking PKG
224 INFO: Bootloader /Users/loretoparisi/Documents/Projects/AI/indic-trans/.venv/lib/python2.7/site-packages/PyInstaller/bootloader/Darwin-64bit/run
224 INFO: checking EXE
225 INFO: Rebuilding out00-EXE.toc because __init__ missing
225 INFO: Building EXE from out00-EXE.toc
225 INFO: Appending archive to EXE /Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans/dist/__init__
230 INFO: Fixing EXE for code signing /Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans/dist/__init__
234 INFO: Building EXE from out00-EXE.toc completed successfully.

But when I run it I get an import error

Traceback (most recent call last):
  File "indictrans/__init__.py", line 9, in <module>
ValueError: Attempted relative import in non-package
[30629] Failed to execute script __init__

This library is built using Cython via cythonize setup, so another option would be to build a executable embedded module using the --embed Cython option.

My setup.py is the following:

#!/usr/bin/env python

import os

from setuptools import setup
from setuptools.extension import Extension
from Cython.Build import cythonize

import numpy


os.environ['PBR_VERSION'] = '1.2.3'
os.environ['SKIP_WRITE_GIT_CHANGELOG'] = '1'
os.environ['SKIP_GENERATE_AUTHORS'] = '1'


extensions = [
    Extension(
        "indictrans._decode.beamsearch",
        [
            "indictrans/_decode/beamsearch.pyx"
        ],
        include_dirs=[numpy.get_include()]
    ),
    Extension(
        "indictrans._decode.viterbi",
        [
            "indictrans/_decode/viterbi.pyx"
        ],
        include_dirs=[numpy.get_include()]
    ),
    Extension(
        "indictrans._utils.ctranxn",
        [
            "indictrans/_utils/ctranxn.pyx"
        ],
        include_dirs=[numpy.get_include()]
    ),
    Extension(
        "indictrans._utils.sparseadd",
        [
            "indictrans/_utils/sparseadd.pyx"
        ],
        include_dirs=[numpy.get_include()]
    )

]

setup(
    setup_requires=['pbr'],
    pbr=True,
    ext_modules=cythonize(extensions)
)

While it's easy to compile a singe python file with the --embed option , see here for more about this, I do not know how to use the --embed option in the setup.py in order to get rid of all the dependencies in the project.

Answer 1

I'm going to refer to the root package directory with the setup.py file as the package directory, which is indic-trans in your case. I'm referring to the first level source directory as the module directory, which is indic-trans/indictrans in your case.

---

If I understand your setup correctly, you have an issue because of trying to create an .exe on a script within the module directory rather than the package directory. This makes the current directory the internal module directory, and relative references will not work.

The way I resolved this in a similar situation was to create a run.py script in the main package directory (same folder as setup.py ) which imports the package and then runs whatever script you want to run inside the module directory.

You didn't post the __init__.py file (by the way, it is generally frowned upon to have any real code in __init.py__ ...), so I'll assume for the below that you want to run the main() function in the base.py script. In that case, make run.py something like:

# This will import everything from the module __init__.py based on it's "all" setup
# You likely want to limit it to just what is needed or just the script you are targeting
# Or you can use the "import indictrans" format and let __init__.py handle it
from indictrans import * 

def main():
    indictrans.base.main()

if __name__ == '__main__':
    main()

Now you can run just run.py from the command line or use it as a debug target to run your package.

Note that with PyInstaller you can target run.py (or run.spec ) and change the name to something else with the --name argument.

pyinstaller --onefile --name indictrans run.spec

which will create indictrans.exe in the dist directory.

---

Running your package as a module

Note that in the module directory, you can also create a __main__.py file that is basically as copy of run.py and does the same thing, but will be run as a module by the Python executable if it is installed locally.

ie python -m indictrans will run your package using __main__.py as the entrypoint.