32-bit program is unable to capture keystrokes made on a 32-bit process, but able to capture those made on a 64-bit process

Question

My environment details:

• Operating system: Windows 7 Enterprise Service Pack 1 (64-bit Operating System)
• Compiler: Microsoft Visual Studio 2005 (Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86)
• My program main.exe and hook.dll are 32-bit
• Internet Explorer (iexplore.exe) is 64-bit
• Chrome (chrome.exe) is 32-bit

I have written a C++ program called main.exe that makes the following calls:

HOOKPROC callback = (HOOKPROC) GetProcAddress(dll, "keyboardHook");
HHOOK hook = SetWindowsHookEx(WH_KEYBOARD, callback, dll, NULL);

The keyboardHook function is defined in a DLL called hook.dll. This function logs every keystroke on the console as well as a file called out.txt.

I perform the following experiment.

1. Execute main.exe on command prompt.
2. Make Internet Explorer (64-bit) as my active window and press A , B and C on it.
3. Make Chrome (32-bit) as my active window and press X , Y and Z on it.
4. Make command prompt my active window and kill main.exe by pressing Ctrl + C .

I see only the following output on the console.

C:\lab\keyhook>main,exe
hinstDLL: 10000000; fdwReason: 1; lpvReserved: 00000000
); scan code: 28; transition state: 1
nCode: 0; wParam: 65 (A); scan code: 30; transition state: 0
nCode: 0; wParam: 65 (A); scan code: 30; transition state: 1
nCode: 0; wParam: 66 (B); scan code: 48; transition state: 0
nCode: 0; wParam: 66 (B); scan code: 48; transition state: 1
nCode: 0; wParam: 67 (C); scan code: 46; transition state: 0
nCode: 0; wParam: 67 (C); scan code: 46; transition state: 1
nCode: 0; wParam: 17 (◄); scan code: 29; transition state: 0
nCode: 0; wParam: 67 (C); scan code: 46; transition state: 0
hinstDLL: 10000000; fdwReason: 2; lpvReserved: 00000000
hinstDLL: 10000000; fdwReason: 0; lpvReserved: 00000001

And I see only the following output in the file I am logging to.

C:\lab\keyhook>type out.txt
); scan code: 28; transition state: 1
nCode: 0; wParam: 65 (A); scan code: 30; transition state: 0
nCode: 0; wParam: 65 (A); scan code: 30; transition state: 1
nCode: 0; wParam: 66 (B); scan code: 48; transition state: 0
nCode: 0; wParam: 66 (B); scan code: 48; transition state: 1
nCode: 0; wParam: 67 (C); scan code: 46; transition state: 0
nCode: 0; wParam: 67 (C); scan code: 46; transition state: 1
nCode: 0; wParam: 17 (◄); scan code: 29; transition state: 0
nCode: 0; wParam: 67 (C); scan code: 46; transition state: 0

You can see that the keystrokes made on 32-bit Chrome have not been captured either in the console or in the log file.

Why is it that my 32-bit program compiled with a 32-bit C++ compiler succeeds in capturing keystrokes made on 64-bit iexplore.exe but fails to capture keystrokes made on 32-bit chrome.exe?

The SetWindowsHookEx documentation on MSDN seems to indicate that my 32-bit program should be able to capture keystrokes made on other 32-bit programs only.

SetWindowsHookEx can be used to inject a DLL into another process. A 32-bit DLL cannot be injected into a 64-bit process, and a 64-bit DLL cannot be injected into a 32-bit process. If an application requires the use of hooks in other processes, it is required that a 32-bit application call SetWindowsHookEx to inject a 32-bit DLL into 32-bit processes, and a 64-bit application call SetWindowsHookEx to inject a 64-bit DLL into 64-bit processes. The 32-bit and 64-bit DLLs must have different names.

Because hooks run in the context of an application, they must match the "bitness" of the application. If a 32-bit application installs a global hook on 64-bit Windows, the 32-bit hook is injected into each 32-bit process (the usual security boundaries apply). In a 64-bit process, the threads are still marked as "hooked." However, because a 32-bit application must run the hook code, the system executes the hook in the hooking app's context; specifically, on the thread that called SetWindowsHookEx. This means that the hooking application must continue to pump messages or it might block the normal functioning of the 64-bit processes.

Here is the complete code of my main program that invokes SetWindowsHookEx .

// Filename: main.cc
#include <iostream>
#include <windows.h>

int main(int argc, char **argv)
{
    HMODULE dll = LoadLibrary("hook.dll");
    if (dll == NULL) {
        std::cerr << "LoadLibrary error " << GetLastError() << std::endl;
        return 1;
    }

    HOOKPROC callback = (HOOKPROC) GetProcAddress(dll, "keyboardHook");
    if (callback == NULL) {
        std::cerr << "GetProcAddress error " << GetLastError() << std::endl;
        return 1;
    }

    HHOOK hook = SetWindowsHookEx(WH_KEYBOARD, callback, dll, NULL);
    if (hook == NULL) {
        std::cerr << "SetWindowsHookEx error " << GetLastError() << std::endl;
        return 1;
    }

    MSG messages;
    while (GetMessage (&messages, NULL, 0, 0))
    {
        TranslateMessage(&messages);
        DispatchMessage(&messages);
    }
    UnhookWindowsHookEx(hook);
}

Here is the DLL code.

// Filename: hook.cc
#include <iostream>
#include <fstream>
#include <windows.h>

extern "C" __declspec(dllexport)
LRESULT keyboardHook(int nCode, WPARAM wParam, LPARAM lParam)
{
    std::ofstream outputTxt("out.txt", std::ofstream::out | std::ofstream::app);
    outputTxt << "nCode: " << nCode << "; wParam: " << wParam
              <<" (" << char(wParam) << "); scan code: "
              << ((lParam & 0xFF0000) >> 16)
              << "; transition state: " << ((lParam & 0x80000000) >> 31)
              << std::endl;
    outputTxt.close();

    std::cout << "nCode: " << nCode << "; wParam: " << wParam
              <<" (" << char(wParam) << "); scan code: "
              << ((lParam & 0xFF0000) >> 16)
              << "; transition state: " << ((lParam & 0x80000000) >> 31)
              << std::endl;
    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

BOOL WINAPI DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
{
    std::cout << "hinstDLL: " << hinstDLL
              << "; fdwReason: " << fdwReason
              << "; lpvReserved: " << lpvReserved << std::endl;

    return TRUE;
}

This is how I compile this project:

vcvars32.bat
cl /LD hook.cc /link user32.lib
cl main.cc /link user32.lib

This is what the processes look like. See that chrome.exe and main.exe are 32-bit processes whereas instances of iexplore.exe are 64-bit processes.

Can you explain why my observation does not match the MSDN documentation?

Answer 1

The comments by Hans Passant to this question and this answer by manuell helped me to understand and fix my code.

The relative path "out.txt" was the source of all confusion.

My 32-bit program successfully injects its 32-bit hook into 32-bit Chrome. Therefore, the hook runs in Chrome's context and the "out.txt" file is written in Chrome's working directory. Thus, the keystrokes X , Y and Z that were made on Chrome as the active window were logged at "C:\\Program Files (x86)\\Google\\Chrome\\Application\\31.0.1650.63\\out.txt".

C:\>type "C:\Program Files (x86)\Google\Chrome\Application\31.0.1650.63\out.txt"
nCode: 0; wParam: 88 (X); scan code: 45; transition state: 0
nCode: 0; wParam: 88 (X); scan code: 45; transition state: 1
nCode: 0; wParam: 89 (Y); scan code: 21; transition state: 0
nCode: 0; wParam: 89 (Y); scan code: 21; transition state: 1
nCode: 0; wParam: 90 (Z); scan code: 44; transition state: 0
nCode: 0; wParam: 90 (Z); scan code: 44; transition state: 1

This output didn't appear in the console because no console is associated with Chrome.

However, my 32-bit program cannot inject its 32-bit hook into 64-bit Internet Explorer. As a result, the hook executes in my program's context and the "out.txt" file is created in my program's working directory when it logs the keystrokes made on 64-bit Internet Explorer. This output is also logged on the console because there is a console associated with my program. This is what you saw mentioned in my question.

A simple fix to my code would be to log the keystrokes to an absolute path "C:\\\\out.txt" instead of the relative path "C:\\out.txt" in hook.cc, so that I can see keystrokes made on both 32-bit programs and 64-bit programs in the same file.