Running Fixtures but not methods in parallel NUnit

Question

I have got a question regarding test parallelization with parameterized fixtures and methods using NUnit 3.8.0.0, .NET 4.0 and C#.

I am currently developing a test suite that runs tests using each resource in a set of resources. Since initializing these resources is quite time consuming, and the resources do not preserve state, I want to share them between my test cases, in order to improve performance. These resource can not be used simultaneously by multiple tests.

In order to do that, I made a base fixture with a constuctor taking one argument, which makes the fixture point to the right resource. The actual test fixtures subclass from this fixture.

I am having a test fixture that uses both constuctor arguments (passed to a base class) and test case arguments. Based on the parameter, the base class initializes some resources. These resources can't be used at the same time.

Now I am trying to parallelize these test cases in such a way that the tests in the different fixtures generated by the class MyParameterizedTestFixture(V1) , MyParameterizedTestFixture(V2) . can run simultaniously, but tests in the same fixture can not (since the resources can not be used simultaniously). Additionally, different fixture classes may not run in parallel.

My code looks like this (with details abstracted away):

[TestFixture]
[TestFixtureSource(typeof(TestData), "FixtureParams")]
public class MyParameterizedTestFixture : BaseFixture
{
     public MyParameterizedTestFixture(Resource resource) : base (resource) { }

     [Test]
     public void Test1() { /* run test using Resource */ }

     [TestCaseSource("Params")]
     public void TestParameterized(object param) { /* run test using Resource and param */ }
}

I tried to add Parallelizable(ParallelScope.Self)] to my derived fixtures, but that results in tests from different fixtures using the same resource (ie having the same version ) fixture parameter, being run simultaniously (it works when only selecting one fixture).

Using [Parallelizable(ParallelScope.Fixture)] is definately not correct, because it will cause different fixtures to run together. Using [Parallelizable(ParallelScope.Children)] is also not correct, because it will cause different test cases in a fixture to run together, and not the different fixtures from the same class.

Now I was wondering if something like what I want could be archived by using a 'layered' approach (marking fixtures parallelizable some way, and methods in another way), or is it possible to define custom parallel scopes?

Answer 1

NUnit allows you to run any fixture in parallel or not. When a fixture is run in parallel, it may run at the same time as any other parallel fixture. Some way of grouping fixtures that can be run together but not with other fixtures would be a nice feature, but it's not one we have right now

When you have multiple instances of the same fixture - ie generic or parameterized fixtures - NUnit treats those instances exactly the same way as it treats any fixture. That is, if the fixture is parallelizable, the instances may run at the same time as any of the other instances as well as instances of different fixtures.

Similarly, you can run test cases (methods) in parallel or not. If a parallel case is contained in a non-parallel fixture, then it only runs in parallel with other methods of that fixture. If it is contained in a parallel fixture, then it can run at the same time as any other parallel method.

In other words, with the features we presently have, parallelism is basically all or nothing for each test, whether it's a suite, a fixture or a test case. This may change with enhancements in a future release.