Access Violation Calling Non-Exported DLL Function C++

Question

I'm trying to access an non-exported function in a DLL for experimental purposes. Without discussing the obvious reasons why this is a bad idea...

I believe I have worked out correct function prototype, and I can confirm the code is jumping to offset in module I want.

The C++ code is

typedef int (__stdcall *FN)(int,wchar_t *);

const DWORD_PTR funcOffset = 0x6F5B9;

int _tmain(int argc, _TCHAR* argv[])
{
    FN myFunction;
    DWORD dwResult;
    HMODULE hModule=LoadLibrary(L"undocumented.dll");

if (hModule != NULL)
{
    DWORD_PTR funcAddress = (DWORD_PTR)GetProcAddress(hModule, "DllRegisterServer") + funcOffset;
    myFunction = (FN)funcAddress;
    wchar_t input[] = L"someinputdata";
    int result = myFunction(0,input);
    std::cout << "Result: " << result << std::endl;
}
else
{
    dwResult = GetLastError();
    std::cerr << "Failed: " << dwResult << std::endl;
}
return 0;

}

The function jumps to offset, but I get an access violation at mov [esi+8], eax. What can I do to prevent this

mov     edi, edi
push    ebp
mov     ebp, esp
sub     esp, 0Ch
cmp     [ebp+<wchar_t *>], 0
push    esi
mov     esi, ecx
jz      short <some location> ; doesn't jump
push    [ebp+<wchar_t *>]     
call    ds:_wcsdup
mov     [esi+8], eax <- access violation

Ecx is set in call ds:_wcsup:

__wcsdup:
75DBEFA3  mov         edi,edi  
75DBEFA5  push        ebp  
75DBEFA6  mov         ebp,esp  
75DBEFA8  cmp         dword ptr [ebp+8],0  
75DBEFAC  je          __wcsdup+2161Ah (75DE05BDh)  
75DBEFB2  mov         ecx,dword ptr [ebp+8]  
75DBEFB5  push        edi  
75DBEFB6  xor         edi,edi  
75DBEFB8  lea         edx,[ecx+2]  
75DBEFBB  mov         ax,word ptr [ecx]  
75DBEFBE  add         ecx,2  
75DBEFC1  cmp         ax,di  
75DBEFC4  jne         __wcsdup+18h (75DBEFBBh)  
75DBEFC6  sub         ecx,edx  
75DBEFC8  sar         ecx,1  
75DBEFCA  push        ebx  
75DBEFCB  push        2  
75DBEFCD  lea         ebx,[ecx+1]  
75DBEFD0  push        ebx  
75DBEFD1  call        _calloc (75DBBE55h)  
75DBEFD6  mov         edi,eax  
75DBEFD8  pop         ecx  
75DBEFD9  pop         ecx  
75DBEFDA  test        edi,edi  
75DBEFDC  je          __wcsdup+2161Eh (75DE05C1h)  
75DBEFE2  push        dword ptr [ebp+8]  
75DBEFE5  push        ebx  
75DBEFE6  push        edi  
75DBEFE7  call        _wcscpy_s (75DBBB70h)  
75DBEFEC  add         esp,0Ch  
75DBEFEF  test        eax,eax  
75DBEFF1  jne         __wcsdup+5626Fh (75E15212h)  
75DBEFF7  mov         eax,edi  
75DBEFF9  pop         ebx  
75DBEFFA  pop         edi  
75DBEFFB  pop         ebp  
75DBEFFC  ret  

Answer 1

It is clear from the assembly that this code expects a meaningful value in ECX .

mov     edi, edi
push    ebp
mov     ebp, esp
sub     esp, 0Ch
cmp     [ebp+<wchar_t *>], 0
push    esi
mov     esi, ecx  !! ecx has not been written to before here, now it is read

You believe that the function is stdcall . And for an stdcall function, the value of ECX is ill-defined on entry. So I'd say that it is pretty clear that the function is not stdcall . The only obvious sign of an input argument being used is that use of ECX . So I'd say that this is a thiscall member function that passes the this pointer in ECX .

Note that I could not be 100% sure about all of the above. Reverse engineering is hard enough at the best of times and do bear in mind that we are working with just the information that you provided.