Pyinstaller - libmagic not found after compiling app

Question

Problem

I'm writing an application in Python. It works just fine in my python environment, but when I compile it with PyInstaller and try to run the resulting executable, I get the following error: ImportError: failed to find libmagic. Check your installation ImportError: failed to find libmagic. Check your installation

I have a feeling that this has something to do with "python-magic-bin", an install containing the binaries that were required for a module named "magic", because if I uninstall "python-magic-bin" from my Python environment and try to run the app, I get the same exact error. I think that for some reason these binaries aren't being carried to the compiled exe.

If it makes a difference, I installed the "python-magic-bin" from a .whl file. This install added a folder called "libmagic" files libmagic.dll and magic.mgc to the directory that "magic" was installed to.

Question

If I'm right about the problem, how do I get PyInstaller to carry over the binaries that "magic" needs?

Recreate the Problem

The following can be done to recreate the problem:

1. Copy and paste this code into your editor. Save it as a file called "test.py"

   import magic m=magic.MAGIC_NONE print(m)

2. Download "python_magic_bin-0.4.14-py2.py3-none-win32.whl" from this link and use the following commands in the interpreter to install the required libraries to Python 3.6

    >>> pip install pyinstaller >>> pip install python-magic >>> pip install python_magic_bin-0.4.14-py2.py3-none-win32.whl >>> pip install libmagic

3. Open a command prompt in the same directory as that "test.py" file and use the following command to compile the program using pyinstaller:

    > pyinstaller test.py

4. After it's done, move to the newly created /dist/test directory ( cd ./dist/test ) and run the .exe using:

    > ./test.exe

After running it, you should see an error reading: ImportError: failed to find libmagic. Check your installation ImportError: failed to find libmagic. Check your installation and Failed to execute script test

Spec File

This is the spec file I'm using to compile my project.

# -*- mode: python -*-

block_cipher = None


a = Analysis(['main.py'],
             pathex=['D:\\Home_Python\\pytags'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)

pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)

exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='main',
          debug=False,
          strip=False,
          upx=True,
          console=True )

coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               name='main')

Versions

For this project, I'm using:

• Python 3.6.3
• PyInstaller 3.3.1
• Libmagic 1.0
• Python-Magic 0.4.15
• Python-Magic-Bin 0.4.14

Answer 1

I figured the problem out after a couple of days, so I'm going to document my solution here for anyone who may be having the same problem in the future.

The issue is the method that libmagic uses to find the .dll file it needs to work properly.

Head over to Lib/site-packages/magic under your Python installation and copy the file named magic.py and the folder named libmagic into the directory of your Python project. After doing that, open up magic.py in your preferred IDE. If you head over to line 156, you'll see this bit of code:

bin_dist_path = os.path.join(os.path.dirname(__file__), 'libmagic')

This is the line that is causing our problem. It looks for a folder called libmagic in the same directory as the magic.py file. Specifically, the issue here is that the __file__ variable does not work the same when the file is frozen into a .exe file so I replaced __file__ with sys.executable . The resulting line looked like this:

bin_dist_path = os.path.join(os.path.dirname(sys.executable), 'libmagic')

Save the file, compile your program, and copy the libmagic folder into the same directory as the resulting .exe file.

If you run the .exe, everything should now work properly.

If there's anything I could clarify or you have questions, please don't hesitate to ask.

Answer 2

Didn't want to put this in another comment because it wouldn't be so visible...

This works for me. Just change the .spec file to include extra files:

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None

extra_files = [('venv/Lib/site-packages/magic','magic')]

a = Analysis(['main.py'],
             pathex=['D:\\Home_Python\\pytags'],
             binaries=[],
             datas=extra_files,
             hiddenimports=[],

...