如何在 WM_PAINT 处理程序中捕获 assert()？

Question

I have MSVC 2019, vc142 x64, SDK 10.0.18362.0, WINAPI game project, JIT debugging is enabled, _DEBUG is defined. I use assert() from standard lib #include <cassert> . assert(expr) call expands to _wassert . If test code assert(false) is placed wherever excluding WM_PAINT handler, a window with Abort/Retry/Ignore options is shown and it's intended behavior.

But in case I have assert(false) in case WM_PAINT , assert window is not being shown. Program just aborts and writes to stderr. Problem is that a lot of game logic is called from WM_PAINT 's handler (eg Core::Update(dt) ) and I cannot catch any assert produced by my code.

WndProc code:

LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)
    {
    case WM_PAINT:
    {
        assert(false);
    }
    break;
    case WM_DESTROY:
        PostQuitMessage(0);
        break;
    default:
        return DefWindowProc(hWnd, message, wParam, lParam);
    }
    return 0;
}

Program spams this message and does not stop:

Program: ...t\source\repos\Test3\x64\Debug\WindowsProject1.exe
File: C:\Users\b2soft\source\repos\Test3\...\Windows...ct1.cpp
Line: 149

Expression: false

For information on how your program can cause an assertion
failure, see the Visual C++ documentation on asserts

(Press Retry to debug the application - JIT must be enabled)Assertion failed!

I want to have the same debug window with Abort/Retry/Ignore options even if assert() is fired from WM_PAINT

Answer 1

We have to be careful when handling the WM_PAINT message. Windows manager constantly monitors which parts of the screen need to be redrawn and adds this information to window's update region. When the GetMessage or the PeekMessage is called and there is no message with higher priority, the WM_PAINT message is generated for windows with nonempty update region.

The BeginPaint prepares window's device context using update region to limit painting area. After this update region is cleared, so the window can collect new areas for painting even before it starts drawing current ones.

If we omit the BeginPaint call, the update region is not cleared (unles we use alternative solutions as for example ValidateRect ) and WM_PAINT is always ready to be dispatched. This leads to endless stream of WM_PAINT messages with high processor usage. Remember that DefWindowProc handles the WM_PAINT internally so these problems manifests only when you explicitly intercept WM_PAINT messages.

Using the asert in WM_PAINT handler without or before the BeginPaint call, leads to unwanted interaction between message boxes and WM_PAINT messages. Depending on settings assert could display message box which starts new (nested) message loop. When message box build by assert is to be shown WM_PAINT for original window is generated again. This leads to next nested loop again and again. After 32 nested calls MessageBox fails and program is aborted (by call to abort function).

Putting BeginPaint (or any alternative which clears update region) before assert stops recursive loop nesting (at least until next WM_PAINT is dispatched) and assert can display message box correctly.

So if you want to use assert in WM_PAINT handler, put it after BeginPaint or ValidateRect(hWnd, NULL) .

LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
    // assert here could lead to abort

    switch (message)
    {
    case WM_PAINT:
    {
        // assert here could lead to abort

        PAINTSTRUCT ps;
        HDC hdc = BeginPaint(hWnd, &ps);
        // Update region cleared, no WM_PAINT will be generated
        // until some event creates new dirty area

        // assert here have chance to be handled properly
        assert(false);

        EndPaint(hWnd, &ps);

        // assert here have chance to be handled properly
    }
    break;
    case WM_DESTROY:
        PostQuitMessage(0);
        break;
    default:
        return DefWindowProc(hWnd, message, wParam, lParam);
    }

    // assert here have chance to be handled properly

    return 0;
}

Remember this is based on observed behavior not on any specific documentation, so you could get different results depending on settings, runtime version and even debugger being attached or not.

Tests done on Windows 10 with VC2017 x86 Debug build ucrt 10.0.17763.0

Personally I didn't know about this limitation and it looks somewhat bad. Especially for eventual asserts placed at a top of the window procedure. Fortunately attached debugger at least displays errors in the output window and I never run debug builds without debugger attached.

Answer 2

If you do not use the BeginPaint and EndPaint functions, the system will continuously send WM_PAINT messages when it is idle, which will cause some abnormal situations.

So only need to modify the code as follows:

LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
        HDC hdc;
        PAINTSTRUCT ps;
        RECT rect;
    switch (message)
    {
    case WM_PAINT:
        hdc = BeginPaint(hWnd, &ps);
        assert(false);
        EndPaint(hWnd, &ps);
        return 0;
    case WM_DESTROY:
        PostQuitMessage(0);
        break;
    default:
        return DefWindowProc(hWnd, message, wParam, lParam);
    }
    return 0;
}

Output: