Run unit tests in parallel for code using Splat

Question

We develop mobile apps using ReactiveUI, and thus use Splat as a Dependency injection framework. When running our unit tests we decided to let them run in parallel to increase feedback speed in the IDE. We noticed that certain tests failed because our SUT used Splat and thus we used splat to inject mocks in the tests. I am sure this has happened to other teams using Splat, so I would want to know if there is a built-in way to circumvent this obstacle.

public interface IDependency
{
    void Invoke();
}

public class SUT1
{
    private IDependency dependency;
    public SUT1()
    {
        dependency = Locator.CurrentMutable.GetService<IDependency>();      
    }
    public void TestThis()
    {
        dependency.Invoke();
    }
}

public class SUT2
{
   private IDependency dependency;
    public SUT2()
    {
        dependency = Locator.CurrentMutable.GetService<IDependency>();      
    }
    public void TestThisToo()
    {
         dependency.Invoke();
    }
}

public class SUT1_Test()
{
    [Fact]
    public void TestSUT1()
    {
        var dependencyMock = new Mock<IDependency>();
        Locator.CurrentMutable.Register(() => dependencyMock.Object, typeof(IDependency));
        var sut = new SUT1();
        sut.TestThis();
        dependencyMock.Verify(x => x.Invoke(), Times.Once);
    }
}

public class SUT2_Test()
{
    [Fact]
    public void TestSUT2()
    {
        var dependencyMock = new Mock<IDependency>();
        Locator.CurrentMutable.Register(() => dependencyMock.Object, typeof(IDependency));
        var sut = new SUT2();
        sut.TestThisToo();
        dependencyMock.Verify(x => x.Invoke(), Times.Once);
    }
}

If the tests are run in a manner where a new mock is injected before the function is called the verification of invocation will be all kinds of messed up. Disabling parallelization enables us to get correct results everytime, at the expense of time used to run the tests in a sequential order.

Answer 1

Disclaimer: I have no idea what SPLAT is. All what I wrote is a generic knowledge on C#, DI, multithreading, testing, and common pitfalls. There is a slight chance that SPLAT solves it in some brillant way. Non-zero chance. Closer to zero.

---

I guess you are using DI in Service Locator (anti)pattern. I guess that your Locator.CurrentMutable.xxx is a globally-static handy thingie that you can access anywhere and ask it for anything. So, it's screwed up for multithreading*) and/or testing. Period.

Quoting from https://github.com/reactiveui/splat#service-location :

Locator.Current is a static variable that can be set on startup, to adapt Splat to other DI/IoC frameworks. This is usually a bad idea.

So, well, yeah, it's static. Not good.

When you run multiple tests in parallel, they all try to register their own mocks for the same service, and they are bound to intersect . If your DI thingie were reasonable, it would throw an exception. Seems it is not, so probably last registration wins, so test A gets a mock registered by test B. This screws up the Verify, since test A reads from mock B and that mock recorded operations from test B not A, so Verify from A fails.

That is correct behavior.

The error is in your test setup, test running, or, in your DI architecture.

If you insist on using service-locator pattern, at least make it nonstatic. Each test must have its own resolution context. If they are shared, test will intersect and fail.

Easiest solution is drop service-locator pattern. Obviously it's not possible since you tell us you have a large code base.

Another option is to de-staticize the locator. Try to make it so that it can be "contextual", so that every test will have its own separate instance of the service provider/locator. This will solve the problem, as registrations will happen on different instances and will not intersect and overwrite.

If your tests are plain and internally single-threaded, you can achieve that by making the Locator.CurrentMutable or the Locator thread-static or anything like that, like 'async context' or anything you consider better. But you should do it only in tests , as making it thread static in real application will probably break it, since your app was not designed with that in mind.

Finally, instead of playing with the code, or test code, or service provider lifetime, you can try adapting how the tests are run. If Locator.CurrentMutable is globally static and has to stay like that, then...

...then your tests cannot be ran in parallel in the same process (because they will intersect)...

...but that does not prevent you from running them in separate processes .

Get the docs, check out source code, and write yourself a brand new runner for xUnit, that will:

• take the tests to run
• create 5-10-20-100 worker processess (configurable?)
• distribute tests to run over worker processes
• each worker process gets his part of the tests
• each worker process runs his tests one by one, not in parallel

Instead of pre-distributing, you can pool&dequeue them, to make sure that there won't be a case where one process lingers with his 99 tests left because one test took longer, etc. Up to you. The most important thing is, do not parallelize them in-single-process, if your service-locator is global and if you register mocks in it on a per-test basis.

*) yes, I know mutex/etc. But it still is screwed unless it is immutable from the consumer's viewpoint, and here it apparently it is not immutable.