Library (dylib) not loaded - image not found - Python IDE

Question

Basically I'm trying to run some Python code from savReaderWriter module in order to create a.sav file ready to open in IBM SPSS. As a macOS user I needed to run these two lines in the terminal first for the module to work:

echo 'export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos'  >> ~/.bash_profile
echo 'export LC_ALL=en_US.UTF-8'  >> ~/.bash_profile

Below you can see a piece of code I'm trying to run in Python:

import savReaderWriter

savFileName = "someFile.sav"
records = [['Test1', 1, 1], ['Test2', 2, 1]]
varNames = ['var1', 'v2', 'v3']
varTypes = {'var1': 5, 'v2': 0, 'v3': 0}
with savReaderWriter.SavWriter(savFileName, varNames, varTypes, ioUtf8=True) as writer:
    for record in records:
        writer.writerow(record)

My problem is that while running the code in Python through terminal.app works like a charm and a new.sav file appears, trying to execute the very same code in an IDE (tried PyCharm and Spyder) gives me an error:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2847, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-94007b092d47>", line 7, in <module>
    with savReaderWriter.SavWriter(savFileName, varNames, varTypes, ioUtf8=True) as writer:
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/savWriter.py", line 198, in __init__
    super(Header, self).__init__(savFileName, ioUtf8, ioLocale)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/generic.py", line 29, in __init__
    self.spssio = self.loadLibrary()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/generic.py", line 117, in loadLibrary
    spssio = self._loadLibs("macos")
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/generic.py", line 89, in _loadLibs
    return [load(os.path.join(path, lib)) for lib in libs][-1]
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/generic.py", line 89, in <listcomp>
    return [load(os.path.join(path, lib)) for lib in libs][-1]
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos/libicuuc48.1.dylib, 6): Library not loaded: @executable_path/../lib/libicudata48.1.dylib
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos/libicuuc48.1.dylib
  Reason: image not found

The module author was unable to help me on this matter, therefore I would be very glad for any suggestions from this community.

EDIT (added sys.path):

From terminal:

 ['',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages',
 '/Users/mg/mne-python']

From IDE:

['/Applications/PyCharm.app/Contents/helpers/pydev',
 '/Users/mg/Documents/Python/Projects/MD',
 '/Applications/PyCharm.app/Contents/helpers/pydev',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages',
 '/Users/mg/mne-python',
 '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/extensions',
 '/Users/mg/Documents/Python/Projects/MD']

Regards,

MG

Answer 1

Found the solution!

Basically I needed to create symbolic links to every dylib that appeared in the error, examples below:

sudo ln -s /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos/libicudata48.1.dylib /usr/local/lib/libicudata48.1.dylib
sudo ln -s /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos/libicui18n48.1.dylib /usr/local/lib/libicui18n48.1.dylib
sudo ln -s /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos/libspssdio.dylib /usr/local/lib/libspssdio.dylib
sudo ln -s /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/savReaderWriter/spssio/macos/libzlib123spss.dylib /usr/local/lib/libzlib123spss.dylib

So theoretically speaking:

sudo ln -s /path/to/original /path/to/symbolic/link

Regards,

MG

Answer 2

I'll add an answer to this question just to make it more generalizable. Would have preferred a comment but don't have the rep, While Maciej is entirely correct, and his answer helped me find my problem. savReaderWriter has since been updated. As of version 3.4.2 there are now 6.dlyb files to be copied over.

Once you have your path to your folder (path is location of your error message), make sure to list ( ls in a terminal window) all files within that folder before creating your symlinks. Then create symlinks for each .dylib

Thanks again to Maciej for the great answer!

Answer 3

For anyone else who is lazy:

ls /Users/your_user/anaconda/envs/quattro8/lib/python2.7/site-packages/savReaderWriter/spssio/macos/ | xargs -I {} sudo ln -s /Users/your_user/anaconda/envs/quattro8/lib/python2.7/site-packages/savReaderWriter/spssio/macos/{} /usr/local/lib/{}

This links all packages in that repo

Answer 4

You can also specify the path to the libraries within PyCharm, which eliminates the need to create the symbolic links:

From the menubar click "Run" -> "Edit Configurations". In the left pane select the configuration for your python script (usually PyCharm creates a configuration for each.py-file). Insert the path to the libraries (shown in the error message) into "Environment Variables".

Example: DYLD_LIBRARY_PATH=/Users/username/miniconda3/envs/savreader/lib/python3.6/site-packages/savReaderWriter/spssio/macos;

Screenshot of PyCharm Run Configuration

Answer 5

After upgrading to Mac OS Catalina, copying all six dylib files to /usr/local/lib/ no longer work (for me at least). To find out what the correct destination was, I did the following:

1. Open the ctypes/__init__.py referred in the error stack tree
2. Add print(os.getcwd()) to get the working directory. For me, the working directory was /Users/Username/GitHub/myproject
3. Since the path in the dylib files is relative to this path ( @executable_path/../lib/ ), the correct destination for the dylib files was /Users/Username/lib

Answer 6

I am not a Mac person, but are you really using Python 3.6? Does the savReaderWriter support that version? I doubt that the I/o module it uses is built for that version.

If this works via Terminal but not an IDE, check that the Python search paths are the same.