How do you resolve 'hidden imports not found!' warnings in pyinstaller for scipy?

Question

I'm working on using pyinstaller to create an .exe for a python program that uses pandas and sklearn. The pyinstaller process completes and produces the dist folder with the executable as expected. However, when I run the .exe I get module import errors related to sklearn and scipy.

I created a test script (test.py) to test imports, which only imports pandas and sklearn and then prints a success message:

import time
import pandas as pd
import sklearn

def main():
  print('hello world!')
  time.sleep(5)


if __name__ == '__main__':
  main()

I'm aware of pyinstaller hooks and I was able to resolve the pandas errors by adding a hook to the pyinstaller hooks directory. I added similar hooks for sklearn and scipy it looks like they're running, but in the pyinstaller output I'm getting warnings that 'Hidden import "sklearn.utils.sparsetools._graph_validation" not found!' and similar one for '._graph_tools'.

Here's the hook for scipy (hook-scipy.py):

print('loading custome hook for scipy')

from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('scipy') 

Here's a snapshot of the warnings generated from running pyinstaller

Here's a snapshot of the error when running test.exe

I'm working in a virtual environment where pyinstaller, pandas, sklearn, scipy and all dependencies are installed (at least I can get the regular test.py script running in this venv). Using PyInstaller 3.3.1, Python 3.6.4 on Windows 10.10.0.

Any help is appreciated!

Answer 1

I'm working on using pyinstaller to create an .exe for a python program that uses pandas and sklearn. The pyinstaller process completes and produces the dist folder with the executable as expected. However, when I run the .exe I get module import errors related to sklearn and scipy.

I created a test script (test.py) to test imports, which only imports pandas and sklearn and then prints a success message:

import time
import pandas as pd
import sklearn

def main():
  print('hello world!')
  time.sleep(5)


if __name__ == '__main__':
  main()

I'm aware of pyinstaller hooks and I was able to resolve the pandas errors by adding a hook to the pyinstaller hooks directory. I added similar hooks for sklearn and scipy it looks like they're running, but in the pyinstaller output I'm getting warnings that 'Hidden import "sklearn.utils.sparsetools._graph_validation" not found!' and similar one for '._graph_tools'.

Here's the hook for scipy (hook-scipy.py):

print('loading custome hook for scipy')

from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('scipy') 

Here's a snapshot of the warnings generated from running pyinstaller

Here's a snapshot of the error when running test.exe

I'm working in a virtual environment where pyinstaller, pandas, sklearn, scipy and all dependencies are installed (at least I can get the regular test.py script running in this venv). Using PyInstaller 3.3.1, Python 3.6.4 on Windows 10.10.0.

Any help is appreciated!