Junit5 - Run tests in a @Suite with @IncludeTags in a specific and consistent run order

Question

I'm trying to find a way to control the order of tests executed in a suite which includes one or more @Tags

Example:

@Suite
@SelectPackages("com.tests")
@IncludeTags({
        "Tag1",
        "Tag2",
})
public class TestSuite { }


@Tag("Tag1")
public class RunThisFirst  { }


@Tag("Tag2")
public class RunThisLast  { }

The order appears somewhat random, I would really like to specify, at the very least, a test to run last after all other tests before the suite is complete.

Answer 1

You may want to use Class Ordering mechanism from Junit5 available since around version 5.8. Depending on your needs you may want to use straightforward @Order annotation and simply annotate every test class with numeric value that represents its sorting order.

To apply this on a @Suite level, that spans across several classes, you may use something like this:

@Suite
@SelectPackages("com.tests")
@IncludeTags({ "Tag1", "Tag2" })
@ConfigurationParameter(
        key = "junit.jupiter.testclass.order.default",
        value= "org.junit.jupiter.api.ClassOrderer$OrderAnnotation")
public class TestSuite { }

@Order(1)
@Tag("Tag1")
public class RunThisFirst  { }

@Order(999)
@Tag("Tag2")
public class RunThisLast  { }

Unnanotated classes are sorted last, which may be undesirable, since it forces you to manually annotate each and every one. You are then free to use any other of provided implementations of ClassOrderer , or even create your own.

@Suite
@SelectPackages("com.tests")
@IncludeTags({ "Tag1", "Tag2" })
@ConfigurationParameter(
        key = "junit.jupiter.testclass.order.default",
        value= "com.tests.MyCustomOrderer")
public class TestSuite { }

@First
@Tag("Tag1")
public class RunThisFirst  { }

@Tag("Tag2")
public class RunThisAnyTime  { }

@Tag("Tag2")
public class RunThisAnyTimeToo  { }

@Last
@Tag("Tag1")
public class RunThisLast  { }

public class MyCustomOrderer implements  org.junit.jupiter.api.ClassOrderer {
    @Override
    public void orderClasses(ClassOrdererContext context) {
        context.getClassDescriptors().sort((o1, o2) -> Boolean.compare(o2.isAnnotated(First.class), o1.isAnnotated(First.class)));
        context.getClassDescriptors().sort((o1, o2) -> Boolean.compare(o1.isAnnotated(Last.class), o2.isAnnotated(Last.class)));
    }
}

@Target({ ElementType.FIELD, ElementType.METHOD, ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
public @interface First { }

@Target({ ElementType.FIELD, ElementType.METHOD, ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
public @interface Last { }