以随机顺序运行JUnit SuiteClasses

Question

I recently started grouping my JUnit tests into test suites. So far this is working quite good for me. The only gripe I have with it is that the order of the @SuiteClasses annotation determines the order of test execution. I know this is the way how it's intended but I'd like to use test suites only to group tests, not to order them.

Since we are using these tests in an, automated, functional (Selenium) testing environment I don't want the test to always execute in the same order. Does anyone know how to use test suites only for their grouping feature?

Thanks in advance!

I've added my code below:

@RunWith(Suite.class)
@SuiteClasses({Test1.class, Test2.class, Test3.class})
public class TestSuite {
private static ScreenRecorder screenRecorder;

@BeforeClass
public static void setUp() {
    screenRecorder = new ScreenRecorder(1, Data.SCREENSHOT_DIR);
    screenRecorder.startRecording(TestSuite.class.getCanonicalName());
}

@AfterClass
public static void tearDown() throws InterruptedException, CommandException {
    screenRecorder.stopRecording();
}

}

Answer 1

You need to use a specific JUnit runner for all your test cases.

Here is one I wrote some time ago:

public class RandomTestRunner extends BlockJUnit4ClassRunner {

    public RandomTestRunner(Class<?> clazz) throws InitializationError {
        super(clazz);
    }

    @Override
    protected List<FrameworkMethod> computeTestMethods() {
        List<FrameworkMethod> methods = super.computeTestMethods();
        List<FrameworkMethod> newMethods = new ArrayList<>(methods);
        Collections.shuffle(newMethods);
        return newMethods;
    }
}

You then need to add the @RunWith(RandomTestRunner.class) annotation to all the test cases classes that you want to run truly randomly, so Test1 , Test2 and Test3 in your case.

The order of your suites will still be the same: Test1 then Test2 then Test3 , only the tests inside these classes will be random.