.so的编译和链接后的运行时“符号查找错误”

Question

first of all I've searched this one quite a bit and haven't managed to find a question with the same situation.

I'm building a 32 bit plugin as a shared-object on a 64 bit machine and using cpp-redis . cpp-redis built fine and I used a cmake toolchain to force 32 bit, checked the .o files during build and confirmed they are 32 bit.

I compiled and linked my very basic program (consisting of one main source file and an extra lib for talking to my host application since this is a plugin).

Upon running loading the plugin into the host app, I am greeted with:

symbol lookup error: plugins/samp-redis.so: undefined symbol: _ZN9cpp_redis16redis_subscriberC1ERKSt10shared_ptrINS_7network10io_serviceEE

I'm pretty stumped as I've added the cpp-redis dir to the linker path with -L and the library with -l, here's my comp/link lines:

(SDK_DIR is my host app sdk, pretty small and minimal set of .c/h files)

g++ -fpermissive -fPIC -m32 -std=c++11 -c -O3 -w -D LINUX -I$(SDK_DIR) -I$(SDK_DIR)/amx -I/usr/local/include/cpp_redis (source .cpp files...)
g++ -v -Wall -O2 -m32 -fshort-wchar -s -shared -L/usr/local/lib/ -lcpp_redis -o $(OUTFILE) *.o

Which runs fine with no errors, I ran the link stage with -v to check the flags went in properly and it all looks fine.

(added some newlines for readability)

Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure
         -v
         --with-pkgversion='Debian 4.9.2-10'
         --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs
         --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++
         --prefix=/usr
         --program-suffix=-4.9
         --enable-shared
         --enable-linker-build-id
         --libexecdir=/usr/lib
         --without-included-gettext
         --enable-threads=posix
         --with-gxx-include-dir=/usr/include/c++/4.9
         --libdir=/usr/lib
         --enable-nls
         --with-sysroot=/
         --enable-clocale=gnu
         --enable-libstdcxx-debug
         --enable-libstdcxx-time=yes
         --enable-gnu-unique-object
         --disable-vtable-verify
         --enable-plugin
         --with-system-zlib
         --disable-browser-plugin
         --enable-java-awt=gtk
         --enable-gtk-cairo
         --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64/jre
         --enable-java-home
         --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64
         --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-amd64
         --with-arch-directory=amd64
         --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
         --enable-objc-gc
         --enable-multiarch
         --with-arch-32=i586
         --with-abi=m64
         --with-multilib-list=m32,m64,mx32
         --enable-multilib
         --with-tune=generic
         --enable-checking=release
         --build=x86_64-linux-gnu
         --host=x86_64-linux-gnu
         --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.9.2 (Debian 4.9.2-10) 
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.9/32/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32/
    :/lib/../lib32/
    :/usr/lib/../lib32/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../
    :/lib/
    :/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-Wall' '-O2' '-m32' '-fshort-wchar' '-s' '-shared' '-L/usr/local/lib/' '-o' 'samp-redis.so' '-shared-libgcc' '-mtune=generic' '-march=i586'
 /usr/lib/gcc/x86_64-linux-gnu/4.9/collect2
     -plugin /usr/lib/gcc/x86_64-linux-gnu/4.9/liblto_plugin.so
     -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
     -plugin-opt=-fresolution=/tmp/cc0Lo5po.res
     -plugin-opt=-pass-through=-lgcc_s
     -plugin-opt=-pass-through=-lc
     -plugin-opt=-pass-through=-lgcc_s
     --sysroot=/
     --build-id
     --eh-frame-hdr
     -m elf_i386
     --hash-style=gnu
     -shared
     -o samp-redis.so
     -s /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.9/32/crtbeginS.o
     -L/usr/local/lib/
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9/32
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32
     -L/lib/../lib32
     -L/usr/lib/../lib32
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9/../../..
     -lcpp_redis amxplugin2.o amxplugin.o main.o
     -lstdc++
     -lm
     -lgcc_s
     -lc
     -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/4.9/32/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32/crtn.o

Am I misunderstanding the linking stage or statically linking? It has been a while since I did C++... I'm all about Python and Go these days!

---

EDIT:

Compiling with -W,--no-undefined results in this output:

Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure
     -v
     --with-pkgversion='Debian 4.9.2-10'
     --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs
     --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++
     --prefix=/usr
     --program-suffix=-4.9
     --enable-shared
     --enable-linker-build-id
     --libexecdir=/usr/lib
     --without-included-gettext
     --enable-threads=posix
     --with-gxx-include-dir=/usr/include/c++/4.9
     --libdir=/usr/lib
     --enable-nls
     --with-sysroot=/
     --enable-clocale=gnu
     --enable-libstdcxx-debug
     --enable-libstdcxx-time=yes
     --enable-gnu-unique-object
     --disable-vtable-verify
     --enable-plugin
     --with-system-zlib
     --disable-browser-plugin
     --enable-java-awt=gtk
     --enable-gtk-cairo
     --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64/jre
     --enable-java-home
     --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64
     --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-amd64
     --with-arch-directory=amd64
     --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
     --enable-objc-gc
     --enable-multiarch
     --with-arch-32=i586
     --with-abi=m64
     --with-multilib-list=m32,m64,mx32
     --enable-multilib
     --with-tune=generic
     --enable-checking=release
     --build=x86_64-linux-gnu
     --host=x86_64-linux-gnu
     --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.9.2 (Debian 4.9.2-10)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.9/32/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32/
    :/lib/../lib32/
    :/usr/lib/../lib32/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/
    :/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../
    :/lib/
    :/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-Wall' '-O2' '-m32' '-fshort-wchar' '-s' '-shared' '-L/usr/local/lib/' '-o' 'samp-redis.so' '-shared-libgcc' '-mtune=generic' '-march=i586'
 /usr/lib/gcc/x86_64-linux-gnu/4.9/collect2
     -plugin /usr/lib/gcc/x86_64-linux-gnu/4.9/liblto_plugin.so
     -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
     -plugin-opt=-fresolution=/tmp/ccOq8qY8.res
     -plugin-opt=-pass-through=-lgcc_s
     -plugin-opt=-pass-through=-lc
     -plugin-opt=-pass-through=-lgcc_s
     --sysroot=/
     --build-id
     --eh-frame-hdr
     -m elf_i386
     --hash-style=gnu
     -shared
     -o samp-redis.so
     -s /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.9/32/crtbeginS.o
     -L/usr/local/lib/
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9/32
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32
     -L/lib/../lib32
     -L/usr/lib/../lib32
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9
     -L/usr/lib/gcc/x86_64-linux-gnu/4.9/../../..
     -Bstatic
     --no-undefined
     -lcpp_redis amxplugin.o amxplugin2.o main.o
     -lstdc++
     -lm
     -lgcc_s
     -lc
     -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/4.9/32/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib32/crtn.o
/usr/bin/ld: cannot find
     -lgcc_s
/usr/bin/ld: cannot find
     -lgcc_s
collect2: error: ld returned 1 exit status
makefile:15: recipe for target 'build' failed
make: *** [build] Error 1

Apparently it can't find gcc_s whatever that is? s=static?

I tried searching for gcc libs:

$ find /usr/ -name libgcc*
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_eh.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/x32/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/4.9/x32/libgcc_eh.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/x32/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/32/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/4.9/32/libgcc_eh.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/32/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s_32.so
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s_x32.so
/usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc_eh.a
/usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc.a
/usr/lib/x86_64-linux-gnu/libgccpp.so.1
/usr/lib/x86_64-linux-gnu/libgccpp.so.1.0.3

Yet doing a ldconfig search shows files named libgcc.so.1 :

$ sudo /sbin/ldconfig -p | grep libgcc
    libgccpp.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libgccpp.so.1
    libgcc_s.so.1 (libc6,x32) => /usr/libx32/libgcc_s.so.1
    libgcc_s.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libgcc_s.so.1
    libgcc_s.so.1 (libc6) => /usr/lib32/libgcc_s.so.1

Could this be the reason?

And googling the error brings up some results so I'll keep digging...

Answer 1

Even if you build cpp_redis as static library for it to be linkable with shared library it must be compiled with -fPIC flag ie code must be relocatable. Looks like in your case linker silently ignores incompatible code when linking shared lib. You can check if symbols from cpp_redis resolved into your plugin by running nm with your .so file.

According to your update, you should link your plugin with -W,--no-undefined not cpp_redis. Most probably it will fail as symbols from host program would be missing. But you can check if symbols from cpp_redis are listed as missing or not.