Compiling a wrapper for a c library using Cython - Linker cant find .dylib of external c library on OSX

Question

I have written a wrapper in Cython for an integration function from NAG ( https://www.nag.co.uk/content/nag-library-c ) c library. It compiles using the python setup.py build --inplace, where setup file is:

from setuptools import Extension, setup
from Cython.Build import cythonize
import re

def loadMacros(headerFile):
    """ Given a .h file, return dict of touples with macros """
    regex = re.compile("#define +(\w+) *(\w*)")
    with open(headerFile) as f:
        macros = dict(map(lambda x: re.match(regex, x).groups(),
                          [l for l in f if re.match(regex, l)]))
    # Remove recursive entries - Note this is not foolproof..
    # while not set(macros.keys()).isdisjoint(macros.values()):
    #     for k, v in macros.items():
    #         if v in macros:
    #             macros[k] = macros[v]
    return macros


nagHome = "/Users/hfmw1m17/NAG/nlmi627dbl"  # "/opt/NAG/cll6a23dhl"
macros = loadMacros(nagHome + "/lp64/include/nag.h")
macros = list(macros.items())

e = Extension("nag_integrate",
              define_macros=macros,
              sources=["nag_integrate.pyx"],
              include_dirs=[nagHome + "/lp64/include",nagHome + "/lp64/lib"],
              library_dirs=[nagHome + "/lp64/lib"],libraries=["nag_nag"],extra_objects=[
        nagHome+"/lp64/lib/libnag_nag.dylib"],runtime_library_dirs=[nagHome+"/lp64/lib/"],extra_link_args=['-Wl,-rpath']
)


setup(ext_modules=cythonize(e,annotate=True,language_level=3))enter code here

with output:

/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/bin/python3.7 setup.py build_ext --inplace
Compiling nag_integrate.pyx because it changed.
[1/1] Cythonizing nag_integrate.pyx
running build_ext
building 'nag_integrate' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include -arch x86_64 -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include -arch x86_64 -DNAG_H= -DNAG_MICROSOFT_THREAD_SAFE= -DNAG_THREAD_SAFE= -DNULLFN=0 -DNULLDFN=0 -DNAGERR_DEFAULT= -DNAGUSER_DEFAULT= -DNAGCOMM_NULL= -DNAGMESG_DEFAULT= -DE01_DEFAULT= -DE04_DEFAULT= -DG13_DEFAULT= -DH02_DEFAULT= -DINIT_FAIL= -DSET_FAIL= -DINIT_MESG= -DINIT_STREAM= -DRDUMMY= -DIDUMMY= -DINIT2DUMMY= -DVprintf= -DVfprintf= -DVsprintf= -DVscanf= -DVfscanf= -DVstrcpy= -DABS= -DFABS= -DSIGN= -DMAX= -DMIN= -DDROUND= -DROUND= -DSQZABS= -DCONJ= -DVCONJ= -DZMULT= -D_nag_expand= -Dnag_stringize= -I/Users/hfmw1m17/NAG/nlmi627dbl/lp64/include -I/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include/python3.7m -c nag_integrate.c -o build/temp.macosx-10.9-x86_64-3.7/nag_integrate.o
gcc -bundle -undefined dynamic_lookup -L/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/lib -arch x86_64 -L/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/nag_integrate.o /Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib/libnag_nag.dylib -L/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib -L/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib/ -lnag_nag -o build/lib.macosx-10.9-x86_64-3.7/nag_integrate.cpython-37m-darwin.so -Wl,-rpath
copying build/lib.macosx-10.9-x86_64-3.7/nag_integrate.cpython-37m-darwin.so -> 

Process finished with exit code 0

However when i import a function from the.so object created i get the following error:

ImportError: dlopen(/Users/hfmw1m17/WaterTankISM/WaterTankISM/nag_integration/nag_integrate.cpython-37m-darwin.so, 2): Library not loaded: libnag_nag.dylib
  Referenced from: /Users/hfmw1m17/WaterTankISM/WaterTankISM/nag_integration/nag_integrate.cpython-37m-darwin.so
  Reason: image not found

libnag_nag.dylib is a dynamic library produced by NAG.

Using otool -L on the shared object of my wrapper results in:

libnag_nag.dylib (compatibility version 0.0.0, current version 27.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

I think its an issue with the linker not being able to find the dynamic library when compiling. Any suggestions on how to solve this problem?

many thanks

Answer 1

Solved using:

install_name_tool -add_rpath path_to_dylib_directory

after running python setup.py build --inplace.

Would like a way of doing this in the setup.py file if anyone can figure that out.

many thanks