How to debug a buffer overrun in Visual C++ 9?

Question

I have a huge MMC snapin written in Visual C++ 9. Every once in a while when I hit F5 in MMC mmc.exe crashes. If I attach a debugger to it I see the following message:

A buffer overrun has occurred in mmc.exe which has corrupted the program's internal state. Press Break to debug the program or Continue to terminate the program.

For more details please see Help topic 'How to debug Buffer Overrun Issues'.

First of all, there's no How to debug Buffer Overrun Issues topic anywhere.

When I inspect the call stack I see that it's likely something with security cookies used to guard against stack-allocated buffer overruns:

MySnapin.dll!__crt_debugger_hook()  Unknown
MySnapin.dll!__report_gsfailure()  Line 315 + 0x7 bytes C
mssvcr90d.dll!ValidateLocalCookies(void (unsigned int)* CookieCheckFunction=0x1014e2e3, _EH4_SCOPETABLE * ScopeTable=0x10493e48, char * FramePointer=0x0007ebf8)  + 0x57 bytes  C
msvcr90d.dll!_except_handler4_common(unsigned int * CookiePointer=0x104bdcc8, void (unsigned int)* CookieCheckFunction=0x1014e2e3, _EXCEPTION_RECORD * ExceptionRecord=0x0007e764, _EXCEPTION_REGISTRATION_RECORD * EstablisherFrame=0x0007ebe8, _CONTEXT * ContextRecord=0x0007e780, void * DispatcherContext=0x0007e738)  + 0x44 bytes    C
MySnapin.dll!_except_handler4(_EXCEPTION_RECORD * ExceptionRecord=0x0007e764, _EXCEPTION_REGISTRATION_RECORD * EstablisherFrame=0x0007ebe8, _CONTEXT * ContextRecord=0x0007e780, void * DispatcherContext=0x0007e738)  + 0x24 bytes C
ntdll.dll!7c9032a8()    
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] 
ntdll.dll!7c90327a()    
ntdll.dll!7c92aa0f()    
ntdll.dll!7c90e48a()    
MySnapin.dll!IComponentImpl<CMySnapin>::GetDisplayInfo(_RESULTDATAITEM * pResultDataItem=0x0007edb0)  Line 777 + 0x14 bytes C++
// more Win32 libraries functions follow

I have lots of code and no idea where the buffer overrun might occur and why. I found this forum discussion and specifically the advise to replace all wcscpy-like functions with more secure versions like wcscpy_s() . I followed the advise and that didn't get me closer to the problem solution.

How do I debug my code and find why and where the buffer overrun occurs with Visual Studio 2008?

Answer 1

Add /RTCs switch to the compiler. This will enable detection of buffer overruns and underruns at runtime. When overrun will be detected, program will break exactly in place where it happened rather than giving you postmortem message.

If that does not help, then investigate wcscpy_s() calls that you mentioned. Verify that the 'number of elements' has correct value. I recently fixed buffer overrun caused incorrect usage of wcscpy_s(). Here is an example:

const int offset = 10;
wchar_t buff[MAXSIZE];
wcscpy_s(buff + offset, MAXSIZE, buff2); 

Notice that buff + offset has MAXSIZE - offset elements, not MAXSIZE.

Answer 2

I just had this problem a minute ago, and I was able to solve it. I searched first on the net with no avail, but I got to this thread.

Anyways, I am running VS2005 and I have a multi-threaded program. I had to 'guess' which thread caused the problem, but luckily I only have a few.

So, what I did was in that thread I ran through the debugger, stepping through the code at a high level function. I noticed that it always occurred at the same place in the function, so now it was a matter of drilling down.

The other thing I would do is step through with the callstack window open making sure that the stack looked okay and just noting when the stack goes haywire.

I finally narrowed down to the line that caused the bug, but it wasn't actually that line. It was the line before it.

So what was the cause for me? Well, in short-speak I tried to memcpy a NULL pointer into a valid area of memory.

I'm surprised the VS2005 can't handle this.

Anyways, hope that helps. Good luck.

Answer 3

I assume you aren't able to reproduce this reliably.

I've successfully used Rational Purify to hunt down a variety of memory problems in the past, but it costs $ and I'm not sure how it would interact with MMC.

Unless there's some sort of built-in memory debugger you may have to try solving this programmatically. Are you able to remove/disable chunks of functionality to see if the problem manifests itself?

If you have "guesses" about where the problem occurs you can try disabling/changing that code as well. Even if you changed the copy functions to _s versions, you still need to be able to reliably handle truncated data.

Answer 4

I have got this overrun when I wanted to increment a value in a pointer variable like this:

*out_BMask++;
instead
(*out_BMask)++;

where out_BMask was declared as int *out_BMask
If you did something like me then I hope this will help you ;)