dlsym() + RTLD_NEXT doesn't work as expected on Ubuntu 20.04

Question

I'm faced a strange runtime behavior on Ubuntu 20.04 (gcc v 9.3.0) when using dlsym() call.

Please, see below a simple example:

• file test.cpp :

#include <iostream>
#include <dlfcn.h>
#include <execinfo.h>
#include <typeinfo>
#include <string>
#include <memory>
#include <cxxabi.h>
#include <cstdlib>

extern "C"
{
    void __cxa_throw(void *ex, void *info, void (*dest)(void *))
    {
        std::cout << "__cxa_throw() invoked \n";

        static void (*const rethrow)(void *, void *, void (*)(void *)) __attribute__((noreturn))
            = (void (*)(void *, void *, void (*)(void *)))dlsym(RTLD_NEXT, "__cxa_throw");

        std::cout << "addr in lib=" << &rethrow << "\n";

        rethrow(ex, info, dest);

        std::terminate();
    }
}

• file main.cpp:

#include <iostream>

void foo()
{
  throw std::runtime_error("error");
}

int main()
{
    foo();
    return 0;
}

Build these 2 files as following:

g++ -fPIC -std=c++17 test.cpp -g -c -o test.o
g++ -shared ./test.o -o libtest.so
g++ main.cpp -std=c++17 -g -pedantic -L./ -ltest -ldl

Feeding ldd to ./a.out gives :

ldd a.out 
    linux-vdso.so.1 (0x00007ffe01186000)
    /usr/local/lib/AppProtection/libAppProtection.so (0x00007f1dbd738000)
    libtest.so (0x00007f1dbd733000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1dbd708000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1dbd526000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1dbd50b000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1dbd319000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1dbd2f4000)
    libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f1dbd1b7000)
    libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f1dbd18d000)
    libXi.so.6 => /lib/x86_64-linux-gnu/libXi.so.6 (0x00007f1dbd17b000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f1dbd964000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1dbd02c000)
    libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f1dbd024000)
    libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f1dbd01c000)
    libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007f1dbd007000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f1dbcfed000)

We can see that libtest.so is resolved before libstdc++.so . My expectation how this code should work :

• __cxa_throw() is defined in 2 shared libraries : libtest.so and libstdc++.so
• when exception is being thrown in foo() call, the __cxa_throw is resolved from libtest.so and invoked.
• invoking dlsym(RTLD_NEXT, "__cxa_throw"); makes lookup for __cxa_throw further in the list of shared libraries, found in libstdc++.so and invoked.

This works as expected on all platforms except Ubuntu 20.04 where rethrow is referencing the __cxa_throw from libtest.so (but not libstc++.so ) and thus causing endless recursion.

Please assist as I'm puzzled with runtime behavior.

Answer 1

The "app protection" component of the Citrix ICA client installs the library /usr/local/lib/AppProtection/libAppProtection.so and adds an entry for it to /etc/ld.so.preload , causing it to be loaded into every dynamically-linked process. Among other things, this library replaces the dlsym function with its own. (If you're curious how this doesn't just always break everything by going into an infinite loop, see How can I intercept dlsym calls using LD_PRELOAD? . It actually seems like Citrix's code may have been copied and pasted straight from that answer.) The problem is that since RTLD_NEXT depends on being able to inspect the return address, special care is required to avoid breaking that when hooking dlsym , and they didn't take that special care. As a result, RTLD_NEXT will look for the symbol in the next library after libAppProtection.so , instead of in the next library after your code, which causes exactly the problem you ran into.

Here's some choices for what to do about it:

• Complain to Citrix support until they fix their buggy library
• sudo apt-get purge icaclient (and optionally reinstall it afterwards, but choosing no when asked about the app protection component )
• Modify your program to load the real dlsym from libdl.so and then use it for any calls that use RTLD_NEXT

Answer 2

正如评论中提到的，该问题是由 Citrix ICA 客户端完成的钩子引起的，该钩子钩住了dlsym() ，从而导致调用错误的dlsym() 。