Unit test multi thread environment with breakpoints in Visual Studio 2008

Question

I am having a problem while debugging a unit test in Visual Studio 2008. The unit test uses a System.Timers.Timer (within the business logic) to handle a certain delay (1'000 miliseconds). The thread running the test is put to sleep for 5'000 miliseconds.

When running the test on its own, it passes. When running all tests (>120) it sometimes passes and sometimes fails ( not deterministic ). I then tried setting a breakpoint in the mock object, where I set the flag that is asserted in the unit test. Now when I debug the test on its own, I get the phenomenon, that the Assert.IsTrue() fails but the Quick Watch tells me, that it is true:

public class ObjectManagerAccessMock : IObjectManagerAccess
{
    public bool ExecuteCommandWasCalled { get; set; }

    public void ExecuteCommand()
    {
        ExecuteCommandWasCalled = true; //set breakpoint here
    }
}

Does the thread running the test continue to run, when another thread is stopped (through the breakpoint)?

Answer 1

I suspect that what you're seeing while debugging is a side-effect. The assertion fails because it is false, but due to threading concerns the value is true by the time you view it in the debugger.

Increasing the timeout in your test should solve the problem, but beware.

Timers seem like a good idea at first but they can lead to problems later on. Environmental issues, such as other executing processes or running the tests on a slower machine, will drive how the Timers are used. Unfortunately, if you increment the timeout to the meet the needs of the slowest machine you'll end up with a series of tests that are extremely slow.

Rather than use Thread.Sleep in your test, consider looking at ways to make your tests run as fast as the code does. For example, raise an event on your test subject when the operation has completed.

[Test]
public void DemonstrateThatTheTestRunsAsFastAsTheSubjectUnderTest()
{
     var resetEvent = new ManualResetEvent(true);

     // configure our test to listen for the completed event
     var subject = new MyTestSubject();
     subject.OnComplete += (sender,e) => resetEvent.Set();

     // perform the long running asynchronous operation
     subject.DoLongRunningOperation();

     // wait up to 10 seconds
     resetEvent.WaitOne(100000);

     Assert.AreTrue(subject.OperationComplete);
}

In the above, we use a ManualResetEvent which will block execution until the Set operation is called. Also note that I am supplying a timeout value to the reset event so that the operation doesn't run forever. If the timeout is exceeded it's likely that our OperationComplete will still be false, so the test fails.

If you want finer detail to determine if the test timed out, the WaitOne method returns a boolean that indicates whether the operation was successful.

bool completedWithoutTimeout = resetEvent.WaitOne(10000);
Assert.IsTrue(completedWithoutTimeout, "The operation timed out.");

Answer 2

Maybe sometimes your BL takes up more than 4 seconds so it adds up into more than 5 seconds and assertion fails, but by the time you are able to inspect it with Quick Watch operation is complete and call sets value to true.

Im pretty sure 1) assert fails cause value is false at the time of assertion 2) value is true by the time you check it and time between those 2 actions >0