Unit testing with Mocks when SUT is leveraging Task Parallel Libaray

Question

I am trying to unit test / verify that a method is being called on a dependency, by the system under test (SUT).

• The depenedency is IFoo.
• The dependent class is IBar.
• IBar is implemented as Bar.
• Bar will call Start() on IFoo in a new (System.Threading.Tasks.)Task, when Start() is called on Bar instance.

Unit Test (Moq):

    [Test]
    public void StartBar_ShouldCallStartOnAllFoo_WhenFoosExist()
    {
        //ARRANGE

        //Create a foo, and setup expectation
        var mockFoo0 = new Mock<IFoo>();
        mockFoo0.Setup(foo => foo.Start());

        var mockFoo1 = new Mock<IFoo>();
        mockFoo1.Setup(foo => foo.Start());


        //Add mockobjects to a collection
        var foos = new List<IFoo>
                       {
                           mockFoo0.Object,
                           mockFoo1.Object
                       };

        IBar sutBar = new Bar(foos);

        //ACT
        sutBar.Start(); //Should call mockFoo.Start()

        //ASSERT
        mockFoo0.VerifyAll();
        mockFoo1.VerifyAll();
    }

Implementation of IBar as Bar:

    class Bar : IBar
    {
        private IEnumerable<IFoo> Foos { get; set; }

        public Bar(IEnumerable<IFoo> foos)
        {
            Foos = foos;
        }

        public void Start()
        {
            foreach(var foo in Foos)
            {
                Task.Factory.StartNew(
                    () =>
                        {
                            foo.Start();
                        });
            }
        }
    }

Moq Exception:

*Moq.MockVerificationException : The following setups were not matched:
IFoo foo => foo.Start() (StartBar_ShouldCallStartOnAllFoo_WhenFoosExist() in
FooBarTests.cs: line 19)*

Answer 1

@dpurrington & @StevenH: If we start putting this kind of stuff in our code

sut.Start();
Thread.Sleep(TimeSpan.FromSeconds(1)); 

and we have thousands of "unit" tests then our tests start running into the minutes instead of seconds. If you had for example 1000 unit tests, it will be hard to have your tests to run in under 5 seconds if someone has gone and littered the test code base with Thread.Sleep.

I suggest that this is bad practice, unless we are explicitly doing Integration testing.

My suggestion would be to use the System.Concurrency.IScheduler interface from System.CoreEx.dll and inject the TaskPoolScheduler implementation.

This is my suggestion for how this should be implemented

using System.Collections.Generic;
using System.Concurrency;
using Moq;
using NUnit.Framework;

namespace StackOverflowScratchPad
{
    public interface IBar
    {
        void Start(IEnumerable<IFoo> foos);
    }

    public interface IFoo
    {
        void Start();
    }

    public class Bar : IBar
    {
        private readonly IScheduler _scheduler;

        public Bar(IScheduler scheduler)
        {
            _scheduler = scheduler;
        }

        public void Start(IEnumerable<IFoo> foos)
        {
            foreach (var foo in foos)
            {
                var foo1 = foo;  //Save to local copy, as to not access modified closure.
                _scheduler.Schedule(foo1.Start);
            }
        }
    }

    [TestFixture]
    public class MyTestClass
    {
        [Test]
        public void StartBar_ShouldCallStartOnAllFoo_WhenFoosExist()
        {
            //ARRANGE
            TestScheduler scheduler = new TestScheduler();
            IBar sutBar = new Bar(scheduler);

            //Create a foo, and setup expectation
            var mockFoo0 = new Mock<IFoo>();
            mockFoo0.Setup(foo => foo.Start());

            var mockFoo1 = new Mock<IFoo>();
            mockFoo1.Setup(foo => foo.Start());

            //Add mockobjects to a collection
            var foos = new List<IFoo>
                       {
                           mockFoo0.Object,
                           mockFoo1.Object
                       };

            //ACT
            sutBar.Start(foos); //Should call mockFoo.Start()
            scheduler.Run();

            //ASSERT
            mockFoo0.VerifyAll();
            mockFoo1.VerifyAll();
        }
    }
}

This now allows the test to run at full speed without any Thread.Sleep.

Note that the contracts have been modified to accept an IScheduler in the Bar constructor (for Dependency Injection) and the IEnumerable is now passed to the IBar.Start method. I hope this makes sense why I made these changes.

Speed of testing is the first and most obvious benefit of doing this. The second and possibly more important benefit of doing this is when you introduce more complex concurrency to your code, which makes testing notoriously difficult. The IScheduler interface and the TestScheduler can allow you to run deterministic "unit tests" even in the face of more complex concurrency.

Answer 2

Your tests uses too much implementation detail, IEnumerable<IFoo> types. Whenever I have to start testing with IEnumerable it always creates some friction.

Answer 3

Thread.Sleep() is definitely a bad idea. I've read on SO several times over that "Real apps don't sleep". Take that as you will but I agree with that statement. Especially during unit tests. If your test code creates false failures, your tests are brittle.

I wrote some tests recently that properly wait for parallel tasks to finish executing and I thought I would share my solution. I realize this is an old post but I thought it would provide value for those looking for a solution.

My implementation involves modifying the class under test and the method under test.

class Bar : IBar
{
    private IEnumerable<IFoo> Foos { get; set; }
    internal CountdownEvent FooCountdown;

    public Bar(IEnumerable<IFoo> foos)
    {
        Foos = foos;
    }

    public void Start()
    {
        FooCountdown = new CountdownEvent(foo.Count);

        foreach(var foo in Foos)
        {
            Task.Factory.StartNew(() =>
            {
                foo.Start();

                // once a worker method completes, we signal the countdown
                FooCountdown.Signal();
            });
        }
    }
}

CountdownEvent objects are handy when you have multiple parallel tasks executing and you need to wait for completion (like when we wait to attempt an assert in unit tests). The constructor initializes with the number of times it should be signaled before it signals waiting code that processing is complete.

The reason the internal access modifier is used for the CountdownEvent is because I usually set properties and methods to internal when unit tests need access to them. I then add a new assembly attribute in the assembly under test's Properties\\AssemblyInfo.cs file so the internals are exposed to a test project.

[assembly: InternalsVisibleTo("FooNamespace.UnitTests")]

In this example, FooCountdown will wait to be signaled 3 times if there are 3 foo objects in Foos.

Now this is how you wait for FooCountdown to signal processing completion so you can move on with your life and quit wasting cpu cycles on Thread.Sleep().

[Test]
public void StartBar_ShouldCallStartOnAllFoo_WhenFoosExist()
{
    //ARRANGE

    var mockFoo0 = new Mock<IFoo>();
    mockFoo0.Setup(foo => foo.Start());

    var mockFoo1 = new Mock<IFoo>();
    mockFoo1.Setup(foo => foo.Start());


    //Add mockobjects to a collection
    var foos = new List<IFoo> { mockFoo0.Object, mockFoo1.Object };

    IBar sutBar = new Bar(foos);

    //ACT
    sutBar.Start(); //Should call mockFoo.Start()
    sutBar.FooCountdown.Wait(); // this blocks until all parallel tasks in sutBar complete

    //ASSERT
    mockFoo0.VerifyAll();
    mockFoo1.VerifyAll();
}