Need help understanding _set_security_error_handler()

Question

So , I've been reading this article:

http://msdn.microsoft.com/en-us/library/aa290051%28VS.71%29.aspx

And I would like to define my custom handler.However, I'm not sure I understand the mechanics well.What happens after a call is made to the user-defined function ( eg the argument of _set_security_error_handler() ) ? Does the program still terminate afterward ? If that is the case, is it possible to terminate only the current thread(assuming that it is not the main thread of the application).AFAIK, each thread has its own stack , so if the stack of a thread gets corrupted, the rest of the application shouldn't be affected.

Finally, if it is indeed possible to only terminate the current thread of execution, what potential problems could such an action cause?

I'm trying to do all this inside an unmanaged C++ dll that I would like to use in my C# code.

Answer 1

The documentation states:

"After handling a buffer overrun, you should terminate the thread or exit the process because the thread's stack is corrupted"

Given this statement, it would seem that you could indeed simply kill the thread. However, you are correct to ask what problems this could cause. The docs for TerminateThread discuss the following problems that can arise from killing a thread:

• If the target thread owns a critical section, the critical section will not be released.
• If the target thread is allocating memory from the heap, the heap lock will not be released.
• If the target thread is executing certain kernel32 calls when it is terminated, the kernel32 state for the thread's process could be inconsistent.
• If the target thread is manipulating the global state of a shared DLL, the state of the DLL could be destroyed, affecting other users of the DLL

See here: http://msdn.microsoft.com/en-us/library/ms686717(VS.85).aspx

The only "safe" thing to do in this circumstance is to exit the process.