Rebuild a dynamic library upon argument typedef change

Question

Let's assume, I have a C structure, DynApiArg_t .

typedef struct DynApiArg_s {
    uint32_t m1;
    ...
    uint32_t mx;
} DynApiArg_t;

The pointer of this struct is passed as an arg to a function say

void DynLibApi(DynApiArg_t *arg)
{
     arg->m1 = 0;
     another_fn_in_the_lib(arg->mold); /* May crash here. (1) */
}

which is present in a dynamic library, libdyn.so . This API is invoked from an executable via a dlopen/dlsym procedure of invocation.

In case this dynamic library is updated to version 2, where DynApiArg_t now has new member, say m2 , as below:

typedef struct DynApiArg_s {
    uint32_t m1;
    OldMbr_t *mold;
    ...
    uint32_t mx;
    uint32_t m2;
    NewMbr *mnew;
} DynApiArg_t;

Without a complete rebuild of the executable or other libs that call this API via a dlopen/dlsym , everytime this API is invoked, I see the process crashing, due to the some dereference of any member in the struct. I understand accessing m2 may be a problem. But access to member mold like below is seen causing crashes.

typedef void (*fnPtr_t)(DynApiArg_t*);


void DynApiCaller(DynApiArg_t *arg)
{
     void *libhdl = dlopen("libdyn.so", RTLD_LAZY | RTLD_GLOBAL);
     fnPtr_t fptr = dlsym(libhdl, "DynLibApi");
     fnptr(arg); /* actual call to the dynamically loaded API (2) */
}

In the call to the API via fnptr, at line marked (2), when the old/existing members (in v1 of lib, when DynApiCaller was initially compiled) is accessed at (1), it happens to be any garbage value or even NULL at times.

What is the right way to handle such updates without a complete recompilation of the executable everytime the dependant libs are updated?

I've seen libs being named with symliks with version numbers like libsolid.so.4 . Is there something related to this versioning system that can help me? If so can you point me to right documentations for these if any?

Answer 1

There are a number of approaches to solve this problem:

1. Include the API version in the dynamic library name.

   Instead of dlopen("libfoo.so") , you use dlopen("libfoo.so.4") . Different major versions of the library are essentially separate, and can coexist on the same system; so, the package name for that library would be eg libfoo-4 . You can have libfoo.so.4 and libfoo.so.5 installed at the same time. Minor versions, say libfoo-4.2 , install libfoo.so.4.2 , and symlink libfoo.so.4 to libfoo.so.4.2 .

2. Initially define the structures with zero padding (required to be zero in earlier versions of the library), and have the later versions reuse the padding fields, but keeping the structures the same size.

3. Use versioned symbol names. This is a Linux extension, using dlvsym() . A single shared library binary can implement several versions of the same dynamic symbol.

4. Use resolver functions to determine the symbols at load time. This allows eg hardware architecture-optimized variants of functions to be selected at run time, but is less useful with a dlopen() -based approach.

5. Use a structure to describe the library API, and a versioned function to obtain/initialize that API.

   For example, version 4 of your library could implement

     struct libfoo_api { int (*func1)(int arg1, int arg2); double *data; void (*func2)(void); /* ... */ }; 

   and only export one symbol,

     int libfoo_init(struct libfoo_api *const api, const int version); 

   Calling that function would initialize the api structure with the symbols supported, with the assumption that the structure corresponds to the specified version. A single shared library can support multiple versions. If a version is not supported, it can return a failure.

   This is especially useful for plugin-type interfaces (although then the _init function is more likely to call application-provided functionality registering functions, rather than fill in a structure), as a single file can contain optimized functionality for a number of versions, optimized for a number of compatible hardware architectures (for example, AMD/Intel architectures with different SSE/AVX/AVX2/AVX512 support).

Note that the above implementation details can be "hidden" in a header file, making actual C code using the shared library much simpler. It also helps making the same API work across a number of OSes, simply by changing the header file to use the approach that works best on that OS, while keeping the actual C interface the same.