JUnit's @TestMethodOrder annotation not working

Question

I'm having a problem with following integration test

import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;

@SpringBootTest
@ActiveProfiles("test")
@TestMethodOrder(OrderAnnotation.class)
public class FooServiceIT {
    @Test
    @Order(1)
    void testUploadSuccess() { ... }
    @Test
    @Order(2)
    void testDownloadSuccess() { ... }
    @Test
    @Order(3)
    void testDeleteSuccess() { ... }
}

I'd expect when I run the test that the execution order would 1, 2, 3 but for some reason, the actual execution order is 2, 3, 1.

Tbh, I'm clueless why the annotation is not working. I'm using Spring Boot 2.1.3 with JUnit 5.4.

Answer 1

You need to configure correctly your IDE.

Requirements

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.4.0</version>
</dependency>

Do not use JUnit 5 that offers your IDE. If you add it as library, you will get:

No tests found for with test runner 'JUnit 5' 
==================== and this exception ===================
TestEngine with ID 'junit-vintage' failed to discover tests
java.lang.SecurityException: class "org.junit.jupiter.api.TestMethodOrder"'s signer information does not match signer information of other classes in the same package

So just include mentioned dependency only and your code will work as you expect:

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class FooServiceIT {

    @Test
    @Order(1)
    public void testUploadSuccess() {
        System.out.println("1");
    }

    @Test
    @Order(2)
    public void testDownloadSuccess() {
        System.out.println("2");
    }

    @Test
    @Order(3)
    public void testDeleteSuccess() {
        System.out.println("3");
    }
}

JUnit result:

1
2
3

Answer 2

I have faced the same issue. But, I found the problem where exactly is on my case. Wrongly import the "Order" class.

Wrong One

import org.springframework.core.annotation.Order;

Right one

*import org.junit.jupiter.api.Order;*

Also, please verify the following five classes with proper import

import org.junit.jupiter.api.Assertions;

import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;

import org.junit.jupiter.api.Order;

import org.junit.jupiter.api.Test;

import org.junit.jupiter.api.TestMethodOrder;

Answer 3

If you have/had JUnit 4, check the import for annotation @Test : import org.junit.Test;

For JUnit 5 import should be: import org.junit.jupiter.api.Test;

It was my issue for this question

Answer 4

From the docs https://junit.org/junit5/docs/current/api/org/junit/jupiter/api/TestMethodOrder.html

 @TestMethodOrder(MethodOrderer.OrderAnnotation.class)
 class OrderedTests {

     @Test
     @Order(1)
     void nullValues() {}

     @Test
     @Order(2)
     void emptyValues() {}

     @Test
     @Order(3)
     void validValues() {}
 }

Answer 5

you need to configure correctly your pom.xml. See mine :

<properties>
    <!-- Dependency versions -->
    <junit.jupiter.version>5.6.0</junit.jupiter.version>
    <maven-surefire-plugin.version>3.0.0-M4</maven-surefire-plugin.version>

    <!-- Java 10 -->
    <maven.compiler.source>1.10</maven.compiler.source>
    <maven.compiler.target>1.10</maven.compiler.target>

    <!-- Encoding -->
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>

<!-- Jupiter API for writing tests -->
<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>${junit.jupiter.version}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<!-- Maven Surefire plugin to run tests -->
<build>
    <plugins>
        <!-- plugin to run test cases from maven -->
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin.version}</version>
        </plugin>
        <!-- Maven plugin to use perticular java version to compile code -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>${maven.compiler.source}</source>
                <target>${maven.compiler.target}</target>
            </configuration>
        </plugin>
    </plugins>
</build>

Now your Junit 5 annotations must function correctly

Answer 6

After applying all the settings based on previous recommended solution. I'm still having the same reverse or random @order execution.

Pom.xml

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.5.3.RELEASE</version>
</parent>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.4.0</version>
    <scope>test</scope>
</dependency>

Maven Dependency Tree:

mvn dependency:tree -Dverbose -Dincludes=org.junit.jupiter:junit-jupiter-engine

[INFO] com.personalitytest.demo:personalitytest:jar:1.0-SNAPSHOT
[INFO] \- org.junit.jupiter:junit-jupiter-engine:jar:5.4.0:test

JUnit Test:

@SpringBootTest
@ActiveProfiles("test")
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class JUnitOrderTest {

    private static final Logger log = LoggerFactory.getLogger(JUnitOrderTest.class);

    @Test
    @Order(1)
    public void testUploadSuccess() {
        log.info("Junit - Order 1");
    }

    @Test
    @Order(2)
    public void testDownloadSuccess() {
        log.info("Junit - Order 2");
    }

    @Test
    @Order(3)
    public void testDeleteSuccess() {
        log.info("Junit - Order 3");
    }
}

Output:

Running com.personalitytest.demo.security.JUnitOrderTest
08:48:35.470 [main] INFO com.personalitytest.demo.security.JUnitOrderTest - Junit - Order 2
08:48:35.480 [main] INFO com.personalitytest.demo.security.JUnitOrderTest - Junit - Order 3
08:48:35.481 [main] INFO com.personalitytest.demo.security.JUnitOrderTest - Junit - Order 1

Answer 7

same issue.

For my case. First, it's the @Test annotation, should be org.junit.jupiter.api.Test , not org.junit.Test .

Second, I added @RunWith(SpringRunner.class) in the head of the test class, after I remove it, the order of test cases run as expected.

Answer 8

For example, by using Spring Boot you can use @FixMethodOrder(MethodSorters.JVM) instead of @TestMethodOrder(MethodOrderer.OrderAnnotation.class) . All of tests are run in the order they appear.

@FixMethodOrder(MethodSorters.JVM)
public class MyTest{

   @Test
   public void zzzz(){}

   @Test
   public void cccc(){}

   @Test
   public void aaaa(){}

   @Test
   public void bbbb(){}
}

Order execution is:

zzzz()
cccc()
aaaa()
bbbb()