ld library path not working under OS X 10.9

Question

I've been trying to figure out why g++ cannot link a program with the armadillo library. The problem is simple:

macbook-pro:arma-xc jmlopez$ g++-4.9 inputs-arma.cpp -larmadillo
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status

Perhaps I did specify where the library should be

macbook-pro:arma-xc jmlopez$ ls -l /usr/lib/libarma*
-rwxr-xr-x  1 root  wheel  37928 Aug 18 14:30 /usr/lib/libarmadillo.4.32.2.dylib
lrwxr-xr-x  1 root  wheel     25 Aug 18 14:30 /usr/lib/libarmadillo.4.dylib -> libarmadillo.4.32.2.dylib
lrwxr-xr-x  1 root  wheel     20 Aug 18 14:30 /usr/lib/libarmadillo.dylib -> libarmadillo.4.dylib

Let us try again

macbook-pro:arma-xc jmlopez$ g++-4.9 -L/usr/lib inputs-arma.cpp -larmadillo
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status

Ok, that did not fix it. Let us see what is going on, for this we will use the -v option which will generate some long output.

macbook-pro:arma-xc jmlopez$ g++-4.9 -L/usr/lib inputs-arma.cpp -larmadillo -v
Using built-in specs.
COLLECT_GCC=g++-4.9
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/lto-wrapper
Target: x86_64-apple-darwin13.3.0
Configured with: ../configure --build=x86_64-apple-darwin13.3.0 --prefix=/usr/local/Cellar/gcc/4.9.1 --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-4.9 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-cloog=/usr/local/opt/cloog --with-isl=/usr/local/opt/isl --with-system-zlib --enable-version-specific-runtime-libs --enable-libstdcxx-time=yes --enable-stage1-checking --enable-checking=release --enable-lto --disable-werror --with-pkgversion='Homebrew gcc 4.9.1' --with-bugurl=https://github.com/Homebrew/homebrew/issues --enable-plugin --disable-nls --enable-multilib --with-native-system-header-dir=/usr/include --with-sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk
Thread model: posix
gcc version 4.9.1 (Homebrew gcc 4.9.1) 
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.3' '-L/usr/lib' '-v' '-shared-libgcc' '-mtune=core2'
 /usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/cc1plus -quiet -v -D__DYNAMIC__ inputs-arma.cpp -fPIC -quiet -dumpbase inputs-arma.cpp -mmacosx-version-min=10.9.3 -mtune=core2 -auxbase inputs-arma -version -o /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//ccApA95Q.s
GNU C++ (Homebrew gcc 4.9.1) version 4.9.1 (x86_64-apple-darwin13.3.0)
    compiled by GNU C version 4.9.1, GMP version 6.0.0, MPFR version 3.1.2-p8, MPC version 1.0.2
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/local/include"
ignoring nonexistent directory "/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../../../x86_64-apple-darwin13.3.0/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Users/jmlopez/github/excentury/excentury/extern/include
 /usr/include
 /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include/c++
 /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include/c++/x86_64-apple-darwin13.3.0
 /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include/c++/backward
 /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include
 /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/include-fixed
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks
End of search list.
GNU C++ (Homebrew gcc 4.9.1) version 4.9.1 (x86_64-apple-darwin13.3.0)
    compiled by GNU C version 4.9.1, GMP version 6.0.0, MPFR version 3.1.2-p8, MPC version 1.0.2
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 5a051bcf2be886a5c6eb8ba33f338693
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.3' '-L/usr/lib' '-v' '-shared-libgcc' '-mtune=core2'
 as -arch x86_64 -force_cpusubtype_ALL -o /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//cchm9rVF.o /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//ccApA95Q.s
COMPILER_PATH=/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/
LIBRARY_PATH=/usr/lib/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/:/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../../:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib/
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.9.3' '-L/usr/lib' '-v' '-shared-libgcc' '-mtune=core2'
 /usr/local/Cellar/gcc/4.9.1/libexec/gcc/x86_64-apple-darwin13.3.0/4.9.1/collect2 -dynamic -arch x86_64 -macosx_version_min 10.9.3 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -weak_reference_mismatches non-weak -o a.out -L/usr/lib -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1 -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../.. -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//cchm9rVF.o -larmadillo -lstdc++ -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
collect2 version 4.9.1
/usr/bin/ld -dynamic -arch x86_64 -macosx_version_min 10.9.3 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -weak_reference_mismatches non-weak -o a.out -L/usr/lib -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1 -L/usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/../../.. -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib /var/folders/6x/ss29s2r51z5cv0_g1ffx0c0c0000gn/T//cchm9rVF.o -larmadillo -lstdc++ -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
@(#)PROGRAM:ld  PROJECT:ld64-236.4
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 armv6m armv7m armv7em
Library search paths:
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
    /usr/local/Cellar/gcc/4.9.1/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
Framework search paths:
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status

The important information is here:

Library search paths:
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
    /usr/local/Cellar/gcc/4.9.1/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib

Somehow it is not looking in /usr/lib . I have tried specifying this directory by first exporting the variable LD_LIBRARY_PATH

macbook-pro:arma-xc jmlopez$ export LD_LIBRARY_PATH=/usr/lib
macbook-pro:arma-xc jmlopez$ g++-4.9 -L/usr/lib inputs-arma.cpp -larmadillo
ld: library not found for -larmadillo
collect2: error: ld returned 1 exit status

No luck. Finally, thinking that ld might be broken I did this:

macbook-pro:arma-xc jmlopez$ ln -s /usr/lib/libarmadillo.dylib /usr/local/Cellar/gcc/4.9.1/lib/libarmadillo.dylib
macbook-pro:arma-xc jmlopez$ g++-4.9 inputs-arma.cpp -larmadillo

Now it works. I'm not sure if there is something wrong with my system but it seems that I have no way of specifying other paths for ld to look for my libraries. Instead I have to use any of the following:

Library search paths:
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
    /usr/local/Cellar/gcc/4.9.1/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib

Is there a way to fix this? Notice that I'm using the g++ from homebrew since I thought that the one that came with xcode was broken. In any case, no luck, at least the homebrew g++ told me the paths where the library was being searched.

Update:

I decided to see if I could add another path:

g++-4.9 -L/Users/jmlopez/ -L/Users/jmlopez/Desktop -L/usr/lib inputs-arma.cpp -larmadillo -v

Now the list of paths where it searches for the libraries is:

Library search paths:
    /Users/jmlopez/
    /Users/jmlopez/Desktop
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /usr/local/Cellar/gcc/4.9.1/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1
    /usr/local/Cellar/gcc/4.9.1/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib

Where is /usr/lib ? What is so special about this directory that ld refuses to look in this directory. It searches in my home directory and in my Desktop. I tried adding /usr/local and it adds it to the list of directories. I'm now starting to wonder about the permissions on the directories... Any ideas?

Answer 1

It is likely that your homebrew g++ was compiled with a custom prefix. The prefix for gcc is usually /usr, meaning that it will look for binaries in /usr/bin, headers in /usr/include and libraries in /usr/lib. If you've compiled with a custom prefix (/usr/local/Cellar/gcc/4.9.1/ in this case, it seems) the it won't look in /usr/lib.

LD_LIBRARY_PATH is how you tell the runtime linker where to look for libraries. So if you had linked to /usr/my/bizarre/lib/path/libmylib.so, then you run it like this:

> LD_LIBRARY_PATH=/usr/my/bizarre/lib/path myprog

To tell g++ where to find libraries, you use the -L command-line option. So your command-line should look like this:

g++-4.9 inputs-arma.cpp -L/usr/lib -larmadillo

Answer 2

I ran into this thread trying to answer a similar question myself. Hopefully this will help out.

You need ld_library_path for the homebrew compiler to work and dyld_library_path for os x to appropriately use the library.

put the following exports somewhere where they can get sourced.

export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH

export DYLD_LIBRARY_PATH=$LD_LIBRARY_PATH:$DYLD_LIBRARY_PATH

This worked for me when compiling with homebrew gcc on OS X 10.9 and 10.10.

Answer 3

The reason for this is that Xcode sets the SDK (in your case the macOS SDK) as the virtual root. So all search paths of the compiler and the linker are being interpreted as relative to the SDK folder.

Specifically, the Linker will not look in /usr/lib, it will look in [macOS SDK]/usr/lib.

If you need to use /usr/lib (eg when linking against OpenSSL's libssl which only exist in /usr/lib but not in [macOS SDK]/usr/lib, you can set "-isysroot /" as the first two "Other Linker Flags" of your project's or target's build settings. This ensures that the Linker will use / as the root (and hence will find libraries in /usr/lib), while the compiler will continue to use the virtual root set by Xcode (ie the macOS SDK).