DLL calling conventions & access violation

Question

after reading and learning for years on this great platform its my first post now.

My Problem:

In C++ I am trying to create a dynamic linked library (32 bit) that will serve as a AQMP Communication Client (based on SimpleAmqpClient). The dll file will then be used inside a third party application (32 bit).

During my tests where I invoke the dll in a custom executable everything works fine. But when I try to use the dll in the third party application I get an access violation error (0x00000000). I found out that the problem may be the function calling convention.

With the few code lines presented below that error can be reproduced. It disappears if I remove the __stdcall expression in mytest.dll. Normally I would expect the code to work because it uses the same calling convention in custom_test.exe and mytest.dll. (Sidenote: the third party application expects a __stdcall function thats why I rely on it)

I would like to understand this behavior. Thanks in advance!

My Setup:

• OS: Windows 7
• 32 bit Compiler: gcc 5.3 (Cygwin)

My Code (custom_test.exe):

#include <stdio.h>
#include <windows.h>


int main(void) {

    HINSTANCE hInstance;    
    hInstance=LoadLibrary("mytest.dll");
    FARPROC lpfnGetProcessID = GetProcAddress(HMODULE(hInstance), "test");

    // Function prototype
    typedef void (__stdcall *myFunction)(void);
    myFunction test;
    test = myFunction(lpfnGetProcessID);

    // Call Function
    test();

    FreeLibrary(hInstance);
}

My Code (mytest.dll):

extern "C" __declspec(dllexport) void __stdcall test(void) {

    printf("Inside Function \n");
}

I compile the code via

• dll: g++ mytest.cpp -o mytest.dll -shared -std=gnu++11
• exe: g++ custom_test.cpp -o custom_test.exe -std=gnu++11

Answer 1

The __stdcall convention makes it the responsibility of the called function to clean up the stack on return, while __cdecl makes it the caller's responsibility.

We can't see the actual declaration in the third-party DLL, but my initial assumption would be that the DLL expects arguments and is either using what it believes to be stack arguments in error, or is cleaning up the stack based on it's assumption of the stack arguments and generally messing with your stack.

EDIT In this instance though, I see that when compiling in 32 bit, the test function is exported with a name of 'test@0'. If you change your GetProcAddress to use this decorated name instead it will work.

Answer 2

Ok now several hours later I can see clear again! Thx IanM_Matrix1 for the suggestion, the so called name decoration was indeed the point.

After my research I can now share some helpful ressources I found: It is important to know that some compilers add different decorations to the function names, see here: http://wyw.dcweb.cn/stdcall.htm

With that in mind one can read this page about Win32 calling conventions in general: http://www.unixwiz.net/techtips/win32-callconv.html

When using gcc the decorations can also be disabled via the flag -Wl,--kill-at .