how to attach to a process that loads my DLL and debug it

Question

basically, I am developing a DLL (an open interface) that will be used by a third party software. I want to debug my DLL after it is loaded by this thirdparty DLL, because I found a specific bug that only happens when my DLL is loaded by this third party software (my DLL works OK for all other software that accepts this open interface)

I set __debugbreak() in my the code DLL and run the third party software. An Visual studio dialog did jump up asking if I want to debug.. but after I click OK, instead of getting get into a visual studio window that shows the line of code that I set a breakpoint, visual studio just crashes.

I found out that the third party softward loads my DLL and runs it in another thread. And I suspect this is the reason that the third party software crashes, because effetively __debugbreak() throws an uncaught execption in a thread launched by the main thread.

In this case, how can i debug my DLL?

Answer 1

It's unfortunate that VS crashes, but you might find a way around that.

Best to start the debugging session right ahead. Or use 'attach to process' when the process is blocked on a messagebox or some other input. Also better use breakpoints.

With __debugbreak() or forced access violation try to ask a new instance of VS not an existing one.