Checking for library existence on MacOS Big Sur

Question

Is there a way of replicating the behaviour of _dyld_shared_cache_contains_path that will work on both MacOS Big Sur as well as MacOS Catalina?

---

My first attempt (using dlopen)

#include <mach-o/dyld.h>
#include <dlfcn.h>
#include <stdio.h>

int library_exists(const char* path) {
    void* handle = dlopen(path, RTLD_LAZY);
    if (handle == NULL) return 0;
    dlclose(handle);
    return 1;
}

int main(void) {
    int result;
    char* path;

    path = "/usr/lib/libc.dylib";
    result = _dyld_shared_cache_contains_path(path);
    printf("_dyld_shared_cache_contains_path(%s) == %d", path, result);
    result = library_exists(path);
    printf("library_exists(%s) == %d", path, result);

    path = "libc.dylib";
    result = _dyld_shared_cache_contains_path(path);
    printf("_dyld_shared_cache_contains_path(%s) == %d", path, result);
    result = library_exists(path);
    printf("library_exists(%s) == %d", path, result);
}

Output:

_dyld_shared_cache_contains_path(/usr/lib/libc.dylib) == 1
library_exists(/usr/lib/libc.dylib) == 1
_dyld_shared_cache_contains_path(libc.dylib) == 0
library_exists(libc.dylib) == 1

This is pretty close, except "libc.dylib" has different behaviour when passed to the 2 functions.

Background

MacOS Big Sur removed shared libraries from the file system, and placed them into a cache instead. The function _dyld_shared_cache_contains_path has been made available in <mach-o/dyld.h> along with this change.

MacOS Big Sur 11.0.1 changenotes

New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

I want a portable binary that checks for the existence of shared libraries on MacOS Catalina or MacOS Big Sur without the need for recompilation for a specific version of MacOS. If we reference _dyld_shared_cache_contains_path and attempt to compile on MacOS Catalina - compilation will fail. I want it to have the same behaviour as _dyld_shared_cache_contains_path .

Answer 1

You can fix dlopen by setting a bunch of environment variables... but you have to spawn a new process.

man dlopen has a more exhaustive description, but basically what you're seeing is a fallback effect because libc.dylib is just a "leaf name" and dyld searches a bunch of paths specified by environment variables for that library.

The following environment variables influence this behaviour:

• DYLD_LIBRARY_PATH
• DYLD_FRAMEWORK_PATH
• DYLD_FALLBACK_LIBRARY_PATH
• DYLD_FALLBACK_FRAMEWORK_PATH
• DYLD_IMAGE_SUFFIX

If not set, then the two FALLBACK ones default to some system paths - the manpage is not quite up to date on this, but the exact values can include the following paths or a subset thereof, based on flags set by the kernel:

• /usr/local/lib
• /usr/lib
• /Library/Frameworks
• /System/Library/Frameworks

DYLD_IMAGE_SUFFIX is a bit different, but if you set it to .dylib , then dlopen("libc") will succeed, so that's unwanted too.

You can disable all fallback behaviour by setting those environment variables to an empty string.

So if you call your code above like DYLD_FALLBACK_LIBRARY_PATH='' ./test , it will work as expected:

_dyld_shared_cache_contains_path(/usr/lib/libc.dylib) == 1
library_exists(/usr/lib/libc.dylib) == 1
_dyld_shared_cache_contains_path(libc.dylib) == 0
library_exists(libc.dylib) == 0

However, if you do a setenv("DYLD_FALLBACK_LIBRARY_PATH", "", 1); in your code, you'll find that to not work. The reason for that is that dyld only looks at environment variables on process initialisation, anything after that is ignored.

Here's a working example with execve :

#include <mach-o/dyld.h>
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

extern char **environ;

int library_exists(const char* path) {
    void* handle = dlopen(path, RTLD_LAZY);
    if (handle == NULL) return 0;
    dlclose(handle);
    return 1;
}

int main(int argc, const char **argv) {
    int result;
    char* path;

    const char *vars[] = { "DYLD_LIBRARY_PATH", "DYLD_FRAMEWORK_PATH", "DYLD_FALLBACK_LIBRARY_PATH", "DYLD_FALLBACK_FRAMEWORK_PATH", "DYLD_IMAGE_SUFFIX" };
    char fail = 0;
    for(size_t i = 0; i < 5; ++i)
    {
        char *v = getenv(vars[i]);
        if(!v || *v != '\0')
        {
            fail = 1;
            setenv(vars[i], "", 1);
        }
    }
    if(fail)
    {
        return execve(argv[0], (char*const*)argv, environ);
    }

    path = "/usr/lib/libc.dylib";
    result = _dyld_shared_cache_contains_path(path);
    printf("_dyld_shared_cache_contains_path(%s) == %d\n", path, result);
    result = library_exists(path);
    printf("library_exists(%s) == %d\n", path, result);

    path = "libc.dylib";
    result = _dyld_shared_cache_contains_path(path);
    printf("_dyld_shared_cache_contains_path(%s) == %d\n", path, result);
    result = library_exists(path);
    printf("library_exists(%s) == %d\n", path, result);
}