Visual studio code binary compatibility

Question

Let's say we have one .exe and multiple .dll's, both of them are written in C / C++ using different Visual studio versions.

.exe and .dll's might have heavy number of third party static libraries, and it's difficult to upgrade whole project (either c++ or dll) to use newer visual studio.

Besides binary compatibility which might raise because of visual studio switching: Visual studio 2015 run-time dependencies or how to get rid of Universal CRT?

There might be even more problems - like debugging and so on: https://randomascii.wordpress.com/2013/09/11/debugging-optimized-codenew-in-visual-studio-2012/

Also as I have understood in older visual studio's there were problems also with C++ function name mangling (which changed from visual studio to visual studio) creating more problems with porting your solution.

1. Allocate / free problem

As I have understood p = malloc() executed in context of .exe (exe is compiled with vs2010) followed with free(p) executed in context of .dll (dll is compiled with vs2013) would simply crash application.

I guess one approach is not to use CRT allocation functions at all (no malloc, free, new, ...) - but to use windows api directly (LocalAlloc, ...) - then code would work across different versions of visual studio's, but overriding all allocation procedures with your own allocation scheme sounds like tedious task.

Do you know any other way for to make vs version mixture possible ?

2. What I have also tested C++ function mangling is compatible between visual studio 2010 and 2013, but is it became a standard now - will it change in incompatible manner in next visual studio versions ?

Answer 1

It's sometimes really useful to ask this kind of questions - you get really useful links from people who comment your question.

I want to copy paste that link here: http://siomsystems.com/mixing-visual-studio-versions/

This is technical background / description for problem.

I by myself tried to prototype some sort of solution to problem #1. (Problem #2 is still bit unclear to me)

The main problem itself comes from dll's MSVCR100.DLL / MSVCR120.DLL / ... which tends to use their own memory management routines instead of trying to have something in common - eg rely on windows api. To my best understanding may be universal CRT introduced in vs2015 is trying to get rid of this problem - however - not sure about it - it requires bit deeper study.

I thought by myself - "ok, if we are loading MSVCR120.DLL - why we cannot intercept malloc/realloc/free functions and route to our CRT". This solution is suitable for exe and dll's which are using "single threaded DLL" or "multi threaded DLL" run-time libraries.

I've picked up minhooks from this site:

http://www.codeproject.com/Articles/44326/MinHook-The-Minimalistic-xx-API-Hooking-Libra

And wrote code snipet like this:

crtCompatibility.cpp:

#include <Windows.h>
#include "MinHook.h"                    //MH_Initialize
#include <map>
#include <vector>
#include <atlstr.h>                     //CStringW
#include <Psapi.h>                      //EnumProcessModules

using namespace std;

map<CStringW, bool> g_dlls;                 // dll file path (lowercased) for all loaded .dll's within process.
map<CStringW, vector<void*> > g_mapHooks;   // dll file path to hooks accosiated with given dll.
map<CStringW, bool> g_myCrtDlls;            // filenames only of all crt's which we enabled by default.
CRITICAL_SECTION g_dllCheck;
bool             g_executingInCrt = false;  // true if executing in dll's crt, don't reroute such mallocs then, otherwise crt gets angry to you.

DWORD g_monitorThread = (DWORD) -1;

#define CRTS_TO_HOOK 10                     // Maximum CRT's to hook
bool hookIsFree[CRTS_TO_HOOK] = { true, true, true, true, true, true, true, true, true, true };

//-------------------------------------------
// malloc rerouter.
//-------------------------------------------

typedef void* (__cdecl *pFuncMalloc) (size_t size);
pFuncMalloc porigMalloc[CRTS_TO_HOOK] = { 0 };
map<CStringW, int> g_AllocationId;

template <int i>
void* __cdecl _thisCrtMalloc( size_t size )
{
    if( !g_executingInCrt && g_monitorThread == GetCurrentThreadId() )
        return malloc( size );

    return porigMalloc[i]( size );
}

pFuncMalloc thisCrtMalloc[CRTS_TO_HOOK] =
{
    _thisCrtMalloc<0>, _thisCrtMalloc<1>, _thisCrtMalloc<2>, _thisCrtMalloc<3>, _thisCrtMalloc<4>, 
    _thisCrtMalloc<5>, _thisCrtMalloc<6>, _thisCrtMalloc<7>, _thisCrtMalloc<8>, _thisCrtMalloc<9>
};

//-------------------------------------------
// realloc rerouter.
//-------------------------------------------

typedef void* (__cdecl *pFuncRealloc) (void* p, size_t size);
pFuncRealloc porigRealloc[CRTS_TO_HOOK] = { 0 };

template <int i>
void* __cdecl _thisCrtRealloc( void* p, size_t size )
{
    if( !g_executingInCrt && g_monitorThread == GetCurrentThreadId() )
        return realloc( p, size );

    return porigRealloc[i]( p, size );
}

pFuncRealloc thisCrtRealloc[CRTS_TO_HOOK] =
{
    _thisCrtRealloc<0>, _thisCrtRealloc<1>, _thisCrtRealloc<2>, _thisCrtRealloc<3>, _thisCrtRealloc<4>,
    _thisCrtRealloc<5>, _thisCrtRealloc<6>, _thisCrtRealloc<7>, _thisCrtRealloc<8>, _thisCrtRealloc<9>
};

//-------------------------------------------
// free rerouter.
//-------------------------------------------

typedef void( __cdecl *pFuncFree ) (void*);
pFuncFree porigFree[CRTS_TO_HOOK] = { 0 };

template <int i>
void __cdecl _thisCrtFree( void* p )
{
    if( !g_executingInCrt && g_monitorThread == GetCurrentThreadId() )
        return free( p );

    porigFree[i]( p );
}

pFuncFree thisCrtFree[CRTS_TO_HOOK] =
{
    _thisCrtFree<0>, _thisCrtFree<1>, _thisCrtFree<2>, _thisCrtFree<3>, _thisCrtFree<4>,
    _thisCrtFree<5>, _thisCrtFree<6>, _thisCrtFree<7>, _thisCrtFree<8>, _thisCrtFree<9>
};


//
//  Normally we could just return true here. But just to minimize amount of hooks
//  enabled accross whole process, we know which plugins are using which visual studio
//  crt.
//
bool CrtNeedsToBeHooked( const wchar_t* pDll )
{
    if( wcsicmp( pDll, L"msvcr120.dll") == 0 )
        return true;

    return false;
}

//
//  Loading one dll might load another (dependent) dll as well.
//  Same is with FreeLibrary. We keep here record of which dll's are loaded
//  to compare with previous state.
//
void EnumDlls( bool bCheckNew )
{
    EnterCriticalSection( &g_dllCheck);
    HMODULE dlls[1024] = { 0 };
    DWORD nItems = 0;
    wchar_t path[MAX_PATH];
    HANDLE hProcess = GetCurrentProcess();

    if( !EnumProcessModules( hProcess, dlls, sizeof( dlls ), &nItems ) )
    {
        LeaveCriticalSection( &g_dllCheck);
        return;
    }

    // Not visited.
    for( auto it = g_dlls.begin(); it != g_dlls.end(); it++ )
        it->second = false;

    nItems /= sizeof( HMODULE );
    for( unsigned int i = 0; i < nItems; i++ )
    {
        path[0] = 0;
        if( !GetModuleFileNameExW( hProcess, dlls[i], path, sizeof( path ) / sizeof( path[0] ) ) )
            continue;

        _wcslwr_s( path, MAX_PATH );

        auto it = g_dlls.find( path );
        if( it != g_dlls.end() )
        {
            // Visited.
            it->second = true;
            continue;
        }

        g_dlls[path] = true;

        if( !bCheckNew )
            continue;

        wchar_t* pDll = wcsrchr( path, L'\\' );
        if( pDll ) pDll++;

        //
        // MSVCRxxx.dll (For example MSVCR100.DLL) is loading, let's hook it's memory allocation routines
        // and route them to our CRT.
        //
        // (P.S. this might be different .dll name for vs2015, haven't tested)
        //
        if( _wcsnicmp( pDll, L"MSVCR", 5 ) == 0 && CrtNeedsToBeHooked(pDll) && g_myCrtDlls.find( pDll ) == g_myCrtDlls.end() )
        {
            // While we are executing our code in hookLoadLibrary, we can execute GetProcLibrary
            // functions, and it's possible to get dead lock because of this.
            // kernel32.dll is waiting for LoadLibrary to complete, but we are waiting for GetProcLibrary
            // to complete.
            LeaveCriticalSection( &g_dllCheck);
            void* f = GetProcAddress( dlls[i], "malloc" );
            void* f2 = GetProcAddress( dlls[i], "realloc" );
            void* f3 = GetProcAddress( dlls[i], "free" );
            EnterCriticalSection( &g_dllCheck);

            int FoundFreeSlot = -1;

            for( int freeSlot = 0; freeSlot < CRTS_TO_HOOK; freeSlot++ )
                if( hookIsFree[freeSlot] == true )
                {
                    FoundFreeSlot = freeSlot;
                    break;
                }

            if( FoundFreeSlot != -1 )
            {
                vector<void*> vecTargets;

                // Hook malloc, realloc, free functions CRT compatibility.
                vecTargets.push_back( f );
                MH_CreateHook( f, thisCrtMalloc[FoundFreeSlot], (void**)&porigMalloc[FoundFreeSlot] );

                vecTargets.push_back( f2 );
                MH_CreateHook( f2, thisCrtRealloc[FoundFreeSlot], (void**)&porigRealloc[FoundFreeSlot] );

                vecTargets.push_back( f3 );
                MH_CreateHook( f3, thisCrtFree[FoundFreeSlot], (void**)&porigFree[FoundFreeSlot] );

                g_mapHooks[path] = vecTargets;
                MH_EnableHook( MH_ALL_HOOKS );
                g_AllocationId[path] = FoundFreeSlot;
                hookIsFree[FoundFreeSlot] = false;
            }
        }
    } //for

    //
    // Check if .dll's were freed.
    //
    for( auto it = g_dlls.begin(); it != g_dlls.end(); )
    {
        if( !it->second )
        {
            // Release trampolines.
            auto hooks = g_mapHooks.find( it->first );
            if( hooks != g_mapHooks.end() )
            {
                // Release allocation slot.
                int allocSlot = g_AllocationId[ it->first ];
                if( allocSlot < CRTS_TO_HOOK )
                    hookIsFree[allocSlot] = true;

                vector<void*>& vec = hooks->second;
                for( size_t i = 0; i < vec.size(); i++ )
                    MH_RemoveHook2( vec[i], false );
            }

            // Dll was freed.
            g_dlls.erase( it++ );
            continue;
        }
        it++;
    } //for

    if( !bCheckNew )
    {
        // Collect CRT names upon which we are running at. .NET might try to draw multiple CRTs.
        for( auto it = g_dlls.begin(); it != g_dlls.end(); it++ )
        {
            CStringW path = it->first;
            wchar_t* pPath = path.GetBuffer( MAX_PATH );
            _wcslwr_s( pPath, MAX_PATH );

            wchar_t* pDll = wcsrchr( pPath, L'\\' );
            if( pDll ) pDll++;

            if( _wcsnicmp( pDll, L"MSVCR", 5 ) == 0 )
                g_myCrtDlls[pDll] = true;
        }
    }

    LeaveCriticalSection( &g_dllCheck );
} //EnumDlls

//-------------------------------------------
// Intercepts LoadLibraryW
//-------------------------------------------

typedef HMODULE( WINAPI *pFuncLoadLibraryW )(const wchar_t* file);
pFuncLoadLibraryW g_origLoadLibraryW = NULL;

HMODULE WINAPI hook_LoadLibraryW( const wchar_t* file )
{
    bool bUpdateLock = g_monitorThread == GetCurrentThreadId();
    if( bUpdateLock )
        g_executingInCrt = true;

    HMODULE h = g_origLoadLibraryW( file );

    if( bUpdateLock )
        g_executingInCrt = false;

    if( !h )
        return h;

    EnumDlls( true );
    return h;
} //hook_LoadLibraryW

//-------------------------------------------
// Intercepts LoadLibraryA
//-------------------------------------------

typedef HMODULE( WINAPI *pFuncLoadLibraryA )(const char* file);
pFuncLoadLibraryA g_origLoadLibraryA = NULL;

HMODULE WINAPI hook_LoadLibraryA( const char* file )
{
    bool bUpdateLock = g_monitorThread == GetCurrentThreadId();
    if( bUpdateLock )
        g_executingInCrt = true;

    HMODULE h = g_origLoadLibraryA( file );

    if( bUpdateLock )
        g_executingInCrt = false;

    if( !h )
        return h;

    EnumDlls( true );
    return h;
} //hook_LoadLibraryW

//-------------------------------------------
// Intercepts FreeLibrary
//-------------------------------------------

typedef BOOL( WINAPI *pFuncFreeLibrary ) (HMODULE h);
pFuncFreeLibrary g_origFreeLibrary;

BOOL WINAPI hook_FreeLibrary( HMODULE h )
{
    bool bUpdateLock = g_monitorThread == GetCurrentThreadId();
    if( bUpdateLock )
        g_executingInCrt = true;

    BOOL b = g_origFreeLibrary( h );

    if( bUpdateLock )
        g_executingInCrt = false;

    if( !b )
        return b;

    EnumDlls( true );
    return b;
} //hook_FreeLibrary

//
//  This function intercepts and starts monitor what new dll's gets loaded and freed.
//  If there is loaded MSVCRxxx.DLL different CRT run-time than we're running in - we intercepts
//  it's memory allocation routines, so allocation in .dll would work identically to allocation in main .exe
//
void EnableCrtMonitor(void)
{
    EnumDlls( false );
    MH_Initialize();
    MH_CreateHookApi( L"kernel32.dll", "LoadLibraryW", hook_LoadLibraryW, (void**)&g_origLoadLibraryW );
    MH_CreateHookApi( L"kernel32.dll", "LoadLibraryA", hook_LoadLibraryA, (void**)&g_origLoadLibraryA );
    MH_CreateHookApi( L"kernel32.dll", "FreeLibrary", hook_FreeLibrary, (void**)&g_origFreeLibrary );
    MH_EnableHook( MH_ALL_HOOKS );
}

class CCrtCompatibilityEnabler
{
public:
    CCrtCompatibilityEnabler()
    {
        InitializeCriticalSection( &g_dllCheck);
        EnableCrtMonitor();
    }

    ~CCrtCompatibilityEnabler()
    {
        MH_DisableHook( MH_ALL_HOOKS );
        MH_Uninitialize();
        DeleteCriticalSection(&g_dllCheck);
    }
} g_CheckCrtShutdown;

//
//  This function enables or disables CRT compatibility hooks.
//
//  Enabling can be done like this:
//      EnableDisableCrtCompatibility( GetCurrentThreadId() );
//  and disabling - running without any argument.
//
//  When hooks are enabled - for thread which is monitored - all memory allocations
//  will be performed using main .exe CRT.
//
void EnableDisableCrtCompatibility( DWORD monitorThread = (DWORD) -1)
{
    g_monitorThread = monitorThread;
}

I however has multiple problems when I wrote this code, so if there are some problems - they are not necessarily easy to fix - it works however with my own code (which is relatively large code base with over 3-4 CRT's which I've got hooked up).

When debugging I've got memory free callback _thisCrtFree from memory which was not allocated by CRT - I suspect that MSVCRxxx.DLL can allocate memory internally, and still call free to that allocated memory - I've decided not to fight with .NET threads (I have mixed mode C++ code in use) - but to specify exact thread which will perform allocation / reallocation / free.

Do enabling CRT compatibility mode can be done like this:

EnableDisableCrtCompatibility( GetCurrentThreadId() );

<call to dll>
    p = malloc(1024);
<back to exe>

EnableDisableCrtCompatibility( );

free( p );

So memory can be now allocated and freed across .dll/.exe / vs version boundary.

But please notice that higher level classes - like vector / string are not guaranteed to be backwards compatible between visual studio versions - so you need to test that separately. If you however have your own string/array classes - you can control code binary compatibility on your own.

Please notice also that if you have some globals which are initialized during .dll startup - then memory management is not hooked and still old CRT malloc/realloc/free are in use. (See in code - g_executingInCrt)

Please feel free to use this code, ask me if you have any questions.

Slightly modified version of minhooks can be found here:

http://www.saunalahti.fi/~tarmpika/codesamples/MinHook.zip

Update 1.5.2016 Approach above was not working with delay loaded dll's - since .dll was loaded at some particular function call (not in LoadLibrary) - and crt initialization again - did not work. I've updated code so that .dll was responsible for enabling or disabling crt compatibility. But code is almost identical to what I've wrote above. If you need also to check constructor / destructor state - it makes sens to wrap up code with scope operators. For example - like this:

void UseExeCrt( bool bExesCrt )
{
    if( bExesCrt )
        EnableDisableCrtCompatibility( GetCurrentThreadId() );
    else
        EnableDisableCrtCompatibility( );
}

... class implemented in .exe, declaration for .dll:

class __declspec(dllimport) CMyOwnClass
{
public:
    void GetComplexData();

    ...
};

... data resides in .dll:

vector<String> g_someDllsData;


... function resides in .dll:

    ...
    UseExeCrt( true );                  // Use exe's CRT for allocating CMyOwnClass.
    {
        CMyOwnClass tempClass;
        tempClass.GetComplexData();     // Since function resides in .exe - we use .exe's CRT.

        UseExeCrt( false );             // Now we use some data which resides in .dll, and we must use .dll's crt.

        g_someDllsData.resize( ... );


        UseExeCrt( true );              // Before deleting CMyOwnClass - we must use again .exe's CRT
    }   //~CMyOwnClass is called.

    UseExeCrt( false );                 // Now back to .dll's CRT. (Just to be safe).

But basic rule is to minimize main exe crt execution time - just to be safe.

18.5.2016 Update As turned out - hook in LoadLibrary + GetProcAddress can hang up in dead lock when complex application is loading and many CRT's are being hooked up. The main problem is that if you have hooked LoadLibrary, you should not use functions of similar api with critical sections. kernel32.dll is waiting for you in one thread, and you're waiting for kernel32.dll in another thread. I have now minimized potential dead lock situation by hooking only what I need (See function CrtNeedsToBeHooked) and also by releasing critical section during GetProcAddress calls.

But it's quite typically if you have hooked some api, you need to take care of multistate / multithread state machine, which api & api hook introduces - problems might be quite non-trivial to reproduce (my fast pc and even on virtual machine problem did not occur, but some slower pc this problem appeared).

Theoretically you could assume that you're fully in control of what .dll's your application is loading, but this is not entirely true. For example - there exists some 3-rd party applications - like tortoise svn, which loads as explorer extension, which in a turn is displayed whenever you open browse dialog.

On one machine this .dll is installed, on another - not. Problem might raise on first machine only.