Hooking a function I don't know the parameters to

Question

Lets say there is a DLL A.DLL with a known entry point DoStuff that I have in some way hooked out with my own DLL fakeA.dll such that the system is calling my DoStuff instead. How do I write such a function such that it can then call the same entry point of the hooked DLL ( A.DLL ) without knowing the arguments of the function? Ie My function in fakeA.DLL would look something like

LONG DoStuff(
// don't know what to put here
)
{
    FARPROC pfnHooked;
    HINSTANCE hHooked;
    LONG lRet;

    // get hooked library and desired function

    hHooked = LoadLibrary("A.DLL");
    pfnHooked = GetProcAddress(hHooked, "DoStuff");

    // how do I call the desired function without knowing the parameters?

    lRet = pfnHooked( ??? );

    return lRet;
}

My current thinking is that the arguments are on the stack so I'm guessing I would have to have a sufficiently large stack variable (a big ass struct for example) to capture whatever the arguments are and then just pass it along to pfnHooked ? Ie

// actual arg stack limit is >1MB but we'll assume 1024 bytes is sufficient
typedef struct { char unknownData[1024]; } ARBITARY_ARG; 

ARBITARY_ARG DoStuff(ARBITARY_ARG args){
    ARBITARY_ARG aRet;
    ...
    aRet = pfnHooked(args);

    return aRet;
}

Would this work? If so, is there a better way?

---

UPDATE: After some rudimentary (and non-conclusive) testing passing in the arbitrary block as arguments DOES work (which is not surprising, as the program will just read what it needs off the stack). However collecting the return value is harder as if it's too large it can cause an access violation. Setting the arbitrary return size to 8 bytes (or maybe 4 for x86) may be a solution to most cases (including void returns) however that's still guesswork. If I had some way of knowing the return type from the DLL (not necessarily at runtime) that would be grand.

Answer 1

This should be a comment but the meta answer is yes you can hook the function without knowing the calling convention and arguments, on an x64/x86 platform. Can it be purely done in C? No, it also needs a good deal of understanding of various calling convention and Assembly programming. The hooking framework will have some of it's bits written in Assembly.

Most hooking framework inherently do that by creating a trampoline that redirects the execution flow from the called function's preamble to stub code that is generally independent of the function it is hooking. In user mode you're guaranteed stack to be always present so you can push your own local variables too on the same stack as long as you can pop them and restore the stack to it's original state.

You don't really need to copy the existing arguments to your own stack variable. You can just inspect the stack, definitely read a bit about calling convention and how stacks are constructed on different architectures for various types of invocation in assembly before you attempt anything.

Answer 2

yes, this is possible do generic hooking 100% correct - one common for multiple functions with different arguments count and calling conventions. for both x86/x64 (amd64) platforms.

but for this need use little asm stubs - of course it will be different for x86/x64 - but it will be very small - several lines of code only - 2 small stub procedures - one for filter pre-call and one for post-call. but most code implementation (95%+) will be platform independent and in c++ (of course this possible do and on c but compare c++ - c source code will be larger, ugly and harder to implement)

in my solution need allocate small executable blocks of code for every hooking api (one block per hooked api). in this block - store function name, original address (or to where transfer control after pre-call - this is depended from hooking method) and one relative call instruction to common asm pre-call stub. magic of this call not only that it transfer control to common stub, but that return address in stack will be point to block itself (ok , with some offset, but if we will use c++ and inheritance - it will be exactly point to some base class, from which we derive our executable block class). as result in common precall stub we will be have information - which api call we hook here and then pass this info to c++ common handler.

one note, because in x64 relative call can be only in range [rip-0x80000000, rip+0x7fffffff] need declare (allocate) this code blocks inside our PE in separate bss section and mark this section as RWE . we can not simply use VirtualAlloc for allocate storage, because returned address can be too far from our common precall stub.

in common asm precall stub code must save rcx,rdx,r8,r9 registers for x64 (this is absolute mandatory) and ecx,edx registers for x86. this is need for case if function use __fastcall calling conventions. however windows api for example almost not using __fastcall - only several __fastcall functions exist from thousands of win api (for ensure this and found this functions - go to LIB folder and search for __imp_@ string (this is __fastcall common prefix) and then call c++ common handler, which must return address of original function(to where transfer control) to stub. stub restore rcx,rdx,r8,r9 (or ecx,edx ) registers and jump (but not call !) to this address

if we want filter only pre-call this is all what we need. however in most case need filter (hook) and post-call - for view/modify function return value and out parameters. and this is also possible do, but need little more coding.

for hook post-call obviously we must replace the return address for hooked api. but on what we must change return address ? and where save original return address ? for this we can not use global variable. even can not use thread local ( __declspec( thread ) or thread_local ) because call can be reqursive. can not use volatile register (because it changed during api call) and can not use non-volatile register - because in this case we will be save it,for restore later - but got some question - where ?

only one (and nice) solution here - allocate small block of executable memory ( RWE ) which containing one relative call instruction to common post-call asm stub. and some data - saved original return address, function parameters(for check out parameters in post handler) and function name

here again, some issuer for x64 - this block must be not too far from common post stub (+/- 2GB) - so the best also allocate this stubs in separate .bss section (with the pre-call stubs).

how many need this ret-stubs ? one per api call (if we want control post call). so not more than api calls active at any time. usually say 256 pre-allocated blocks - more than enough. and even if we fail allocate this block in pre-call - we only not control it post call, but not crash. and we can not for all hooked api want control post-call but only for some.

for very fast and interlocked alloc/free this blocks - need build stack semantic over it. allocate by interlocked pop and free by interlocked push. and pre initialize (call instruction) this blocks at begin (while push all it to stack, for not reinitialize it every time in pre-call)

common post-call stub in asm is very simply - here we not need save any registers. we simply call c++ post handler with address of block (we pop it from stack - result of call instruction from block) and with original return value ( rax or eax ). strictly said - api function can return pair rax+rdx or eax+edx but 99.9%+ of windows api return value in single register and i assume that we will be hooking only this api. however if want, can little adjust code for handle this too (simply in most case this not need)

c++ post call handler restore original return address (by using _AddressOfReturnAddress() ), can log call and/or modify out parameters and finally return to.. original caller of api. what our handler return - this and will be final return value of api call. usually we mast return original value.

c++ code

#if 0 
#define __ASM_FUNCTION __pragma(message(__FUNCDNAME__" proc\r\n" __FUNCDNAME__ " endp"))
#define _ASM_FUNCTION {__ASM_FUNCTION;}
#define ASM_FUNCTION {__ASM_FUNCTION;return 0;}
#define CPP_FUNCTION __pragma(message("extern " __FUNCDNAME__ " : PROC ; "  __FUNCTION__))
#else
#define _ASM_FUNCTION
#define ASM_FUNCTION
#define CPP_FUNCTION
#endif

class CODE_STUB
{
#ifdef _WIN64
    PVOID pad;
#endif
    union
    {
        DWORD code;
        struct  
        {
            BYTE cc[3];
            BYTE call;
        };
    };
    int offset;

public:

    void Init(PVOID stub)
    {
        // int3; int3; int3; call stub
        code = 0xe8cccccc;
        offset = RtlPointerToOffset(&offset + 1, stub);

        C_ASSERT(sizeof(CODE_STUB) == RTL_SIZEOF_THROUGH_FIELD(CODE_STUB, offset));
    }

    PVOID Function()
    {
        return &call;
    }

    // implemented in .asm
    static void __cdecl retstub()  _ASM_FUNCTION;
    static void __cdecl callstub() _ASM_FUNCTION;
};

struct FUNC_INFO
{
    PVOID OriginalFunc;
    PCSTR Name;

    void* __fastcall OnCall(void** stack);
};

struct CALL_FUNC : CODE_STUB, FUNC_INFO
{
};  

C_ASSERT(FIELD_OFFSET(CALL_FUNC,OriginalFunc) == sizeof(CODE_STUB));

struct RET_INFO 
{
    union
    {
        struct  
        {
            PCSTR Name;
            PVOID params[7];
        };

        SLIST_ENTRY Entry;
    };

    INT_PTR __fastcall OnCall(INT_PTR r);
};

struct RET_FUNC : CODE_STUB, RET_INFO 
{
};

C_ASSERT(FIELD_OFFSET(RET_FUNC, Entry) == sizeof(CODE_STUB));

#pragma bss_seg(".HOOKS")

RET_FUNC g_rf[1024];//max call count
CALL_FUNC g_cf[16];//max hooks count

#pragma bss_seg() 

#pragma comment(linker, "/SECTION:.HOOKS,RWE")

class RET_FUNC_Manager 
{
    SLIST_HEADER _head;

public:

    RET_FUNC_Manager()
    {
        PSLIST_HEADER head = &_head;

        InitializeSListHead(head);

        RET_FUNC* p = g_rf;
        DWORD n = RTL_NUMBER_OF(g_rf);

        do 
        {
            p->Init(CODE_STUB::retstub);
            InterlockedPushEntrySList(head, &p++->Entry);
        } while (--n);
    }

    RET_FUNC* alloc()
    {
        return static_cast<RET_FUNC*>(CONTAINING_RECORD(InterlockedPopEntrySList(&_head), RET_INFO, Entry));
    }

    void free(RET_INFO* p)
    {
        InterlockedPushEntrySList(&_head, &p->Entry);
    }
} g_rfm;

void* __fastcall FUNC_INFO::OnCall(void** stack)
{
    CPP_FUNCTION;

    // in case __fastcall function in x86 - param#1 at stack[-1] and param#2 at stack[-2]

    // this need for filter post call only
    if (RET_FUNC* p = g_rfm.alloc())
    {
        p->Name = Name;
        memcpy(p->params, stack, sizeof(p->params));
        *stack = p->Function();
    }

    return OriginalFunc;
}

INT_PTR __fastcall RET_INFO::OnCall(INT_PTR r)
{
    CPP_FUNCTION;

    *(void**)_AddressOfReturnAddress() = *params;

    PCSTR name = Name;
    char buf[8];
    if (IS_INTRESOURCE(name))
    {
        sprintf(buf, "#%04x", (ULONG)(ULONG_PTR)name), name = buf;
    }

    DbgPrint("%p %s(%p, %p, %p ..)=%p\r\n", *params, name, params[1], params[2], params[3], r);
    g_rfm.free(this);
    return r;
}

struct DLL_TO_HOOK 
{
    PCWSTR szDllName;
    PCSTR szFuncNames[];
};

void DoHook(DLL_TO_HOOK** pp)
{
    PCSTR* ppsz, psz;
    DLL_TO_HOOK *p;
    ULONG n = RTL_NUMBER_OF(g_cf);

    CALL_FUNC* pcf = g_cf;

    while (p = *pp++)
    {
        if (HMODULE hmod = LoadLibraryW(p->szDllName))
        {
            ppsz = p->szFuncNames;

            while (psz = *ppsz++)
            {
                if (pcf->OriginalFunc = GetProcAddress(hmod, psz))
                {
                    pcf->Name = psz;
                    pcf->Init(CODE_STUB::callstub);

                    // do hook: pcf->OriginalFunc -> pcf->Function() - code for this skiped
                    DbgPrint("hook: (%p) <- (%p)%s\n", pcf->Function(), pcf->OriginalFunc, psz);

                    if (!--n)
                    {
                        return;
                    }

                    pcf++;
                }
            }
        }
    }
}

asm x64 code:

extern ?OnCall@FUNC_INFO@@QEAAPEAXPEAPEAX@Z : PROC ; FUNC_INFO::OnCall
extern ?OnCall@RET_INFO@@QEAA_J_J@Z : PROC ; RET_INFO::OnCall

?retstub@CODE_STUB@@SAXXZ proc
    pop rcx
    mov rdx,rax
    call ?OnCall@RET_INFO@@QEAA_J_J@Z
?retstub@CODE_STUB@@SAXXZ endp

?callstub@CODE_STUB@@SAXXZ proc
    mov [rsp+10h],rcx
    mov [rsp+18h],rdx
    mov [rsp+20h],r8
    mov [rsp+28h],r9
    pop rcx
    mov rdx,rsp
    sub rsp,18h
    call ?OnCall@FUNC_INFO@@QEAAPEAXPEAPEAX@Z
    add rsp,18h
    mov rcx,[rsp+8]
    mov rdx,[rsp+10h]
    mov r8,[rsp+18h]
    mov r9,[rsp+20h]
    jmp rax
?callstub@CODE_STUB@@SAXXZ endp

asm x86 code

extern ?OnCall@FUNC_INFO@@QAIPAXPAPAX@Z : PROC ; FUNC_INFO::OnCall
extern ?OnCall@RET_INFO@@QAIHH@Z : PROC ; RET_INFO::OnCall

?retstub@CODE_STUB@@SAXXZ proc
    pop ecx
    mov edx,eax
    call ?OnCall@RET_INFO@@QAIHH@Z
?retstub@CODE_STUB@@SAXXZ endp

?callstub@CODE_STUB@@SAXXZ proc
    xchg [esp],ecx
    push edx
    lea edx,[esp + 8]
    call ?OnCall@FUNC_INFO@@QAIPAXPAPAX@Z
    pop edx
    pop ecx
    jmp eax
?callstub@CODE_STUB@@SAXXZ endp

you can ask from where i know this decorated names like ?OnCall@FUNC_INFO@@QAIPAXPAPAX@Z ? look for very begin of c++ code - for several macros - and first time compile with #if 1 and look in output window. hope you understand (and you will be probably need use this names, but not my names - decoration can be different)

and how call void DoHook(DLL_TO_HOOK** pp) ? like that:

DLL_TO_HOOK dth_kernel32 = { L"kernel32", { "VirtualAlloc", "VirtualFree", "HeapAlloc", 0 } };
DLL_TO_HOOK dth_ntdll = { L"ntdll", { "NtOpenEvent", 0 } };

DLL_TO_HOOK* ghd[] = { &dth_ntdll, &dth_kernel32, 0 };
DoHook(ghd);

Answer 3

Lets say there is a DLL A.DLL with a known entry point DoStuff

If the entry point DoStuff is known it ought to be documented somewhere, at the very least in some C header file. So a possible approach might be to parse that header to get its signature (ie the C declaration of DoStuff ). Maybe you could fill some database with the signature of all functions declared in all system header files, etc... Or perhaps use debug information if you have it.

If you call some function (in C) and don't give all the required parameters, the calling convention & ABI will still be used, and these (missing) parameters get garbage values (if the calling convention defines that parameter to be passed in a register, the garbage inside that register; if the convention defines that parameter to be passed on the call stack , the garbage inside that particular call stack slot). So you are likely to crash and surely have some undefined behavior (which is scary , since your program might seem to work but still be very wrong).

However, look also into libffi . Once you know (at runtime) what to pass to some arbitrary function, you can construct a call to it passing the right number and types of arguments.

My current thinking is that the arguments are on the stack

I think it is wrong (at least on many x86-64 systems). Some arguments are passed thru registers. Read about x86 calling conventions .

Would this work?

No, it won't work because some arguments are passed thru registers, and because the calling convention depends upon the signature of the called function (floating point values might be passed in different registers, or always on the stack; variadic functions have specific calling conventions; etc....)

BTW, some recent C optimizing compilers are able to do tail call optimizations, which might complicate things.

Answer 4

There is no standard way of doing this because lot of things like calling conventions, pointer sizes etc matter when passing arguments. You will have to read the ABI for your platform and write an implementation, which I fear again won't be possible in C. You will need some inline assembly.

One simple way to do it would be (for a platform like X86_64) -

MyDoStuff:
    jmpq *__real_DoStuff

This hook does nothing but just calls the real function. If you want to do anything useful while hooking you will have to save restore some registers before the call (again what to save depends on the ABI)