pyinstaller error: cannot find scipy (No module named _ufuncs_cxx)

Question

I am using pyinstaller to convert python script into a binary in Ubuntu (14.04). I use Canopy (Enthought) to manage all python libraries.

The code uses networkx, numpy, and scipy. Here is my spec file:

# -*- mode: python -*-
a = Analysis(['main_test.py'],
             pathex=['/home/sean/Desktop/prog',],
             hiddenimports=[],
             hookspath=None,
             runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='main_test',
          debug=False,
          strip=None,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=None,
               upx=True,
               name='main_test')

At first I got the error:

ImportError: libmkl_gf.so: cannot open shared object file: 
    No such file or directory

Then I found the .so library in

/home/sean/Canopy/appdata/canopy-1.3.0.1715.rh5-x86/lib

I manually copied several .so files into the dist direcotry.

However, I got another error:

  File "/home/sean/Enthought/Canopy_32bit/User/lib/python2.7/site-
        packages/PyInstaller/loader/pyi_importers.py", line 409, in load_module
        module = imp.load_module(fullname, fp, filename, self._c_ext_tuple)
  File "_ufuncs.pyx", line 1, in init scipy.special._ufuncs
        (scipy/special/_ufuncs.c:21824)
ImportError: No module named _ufuncs_cxx

How do I fix this error? And how should I modify the spec file to add those libraries and modules?

Edit:

I found the solutuion. My question is now: How can I modify the spec file to add the .so libraies? Now I have to mannually copy a number of .so files to the dist directory...

Edit2

It turns out that I have to add it to COLLECT :

a.binaries + ["libmkl_gf.so" , 
  "/home/sean/Canopy/appdata/canopy-1.3.0.1715.rh5-x86/lib/libmkl_gf.so", 
  "binaries"]

Is there any easy way to find the hidden imports or libraries?

Thanks

Answer 1

I just came out from solving the problem. I had to specify the missing modules with the --hidden-import flag. There were a lot of them missing, but I noticed most of them were from scipy.integrate. So I specified:

pyinstaller --hidden-import=scipy.integrate --hidden-import=scipy.integrate.quadpack --hidden-import=scipy.integrate._vode bla bla bla bla -F --windowed myscript.py

Painful, but worked

Answer 2

Do you want to try adding the library paths into LD_LIBRARY_PATH? something like,

export LD_LIBRARY_PATH=/home/sean/Canopy/appdata/canopy-1.3.0.1715.rh5-x86/lib

or

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/sean/Canopy/appdata/canopy-1.3.0.1715.rh5-x86/lib (if already set up by admin) so that at run time all the .so in that folder won't give you linking error...

Oh I got what you mean,

import sys

sys.path.append('your_lib_path')

This should work.