How to tell if a function is called within a vfork()'ed process?

Question

I have an interposer library for C. It is not possible to interpose vfork() because vfork() (interposing function) cannot return to the calling function after having called the real vfork() . But I am interposing _exit() because I need to know when the process finishes. And of course I am interposing exec*() functions. My problem is that when interposing _exit() there are certain things that I want to do when _exit() is called by a normal process but not when the process is the restricted vfork() 'ed process.

How can I tell in a C program when my process is the vfork() 'ed process and when I do not have access to the process id returned by vfork() ?

Interposer library:

/* COMPILE: gcc -shared -ldl -fPIC -o libso_interposer.so so_interposer.c -std=c99 */
/* RUN:     LD_PRELOAD=./libso_interposer.so so_interposer_test */
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <unistd.h>


static void (*_exit_R)(int) = (void *) 0;
static pid_t (*fork_R)(void) = (void *) 0;
static void teardown_interposer() {
    fprintf(stderr, "Destructing so_interposer.\n");
    /* Concurrency protected code to be executed only once
     * when the process finishes! */
    /* Must not be executed if/when vfork() process finishes. */
}

pid_t fork(void) {
    *(pid_t **) (&fork_R) = dlsym(RTLD_NEXT, "fork");
    /* Code to prepare for a new process.
    * More preparation in exec* interposing functions.*/
    pid_t pid = fork_R();
    return pid;
}
__attribute__((noreturn)) void _exit(int status) {
    *(void **) (&_exit_R) = dlsym(RTLD_NEXT, "_exit");
    fprintf(stderr, "Process '%lld' called _exit(%i).\n", (signed long long int) getpid(), status);
    teardown_interposer();
    _exit_R(status);
}

Testing binary:

/* COMPILE: gcc -std=c99 -D FORK=vfork -o so_interposer_test so_interposer_test.c */
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#if !defined(FORK)
#error "Define FORK: 'fork' or 'vfork'!"
#endif
int main(void) {
    pid_t pid = FORK();
    if(pid == 0) {
        _exit(EXIT_SUCCESS);
    } else if(pid > 0) {
        _exit(EXIT_SUCCESS);
    }
    return EXIT_SUCCESS;
}

Answer 1

For a sufficiently recent version of gcc, you should be able to wrap vfork as:

typedef pid_t (*vfork_t)(void);
extern vfork_t wrap_vfork();

pid_t vfork(void) {
    vfork_t f = wrap_vfork();
    return f();
}

where wrap_vfork does all your vfork wrapping work, and returns a pointer to the real vfork (without calling it). Gcc 6.3.0 (-O3) compiles this as:

    .globl  vfork
    .type   vfork, @function
vfork:
.LFB11:
    .cfi_startproc
    subq    $8, %rsp
    .cfi_def_cfa_offset 16
    xorl    %eax, %eax
    call    wrap_vfork
    addq    $8, %rsp
    .cfi_def_cfa_offset 8
    jmp     *%rax
    .cfi_endproc
.LFE11:
    .size   vfork, .-vfork

The important thing to note is that it jumps directly to the actual vfork function rather than calling it, so does not itself need to return (the real vfork will return directly to this function's caller)

If you're not comfortable relying on the compiler to do the tail-call optimization for you, you can write this one small routine in asm directly rather than in C.