如何让 maven surefire 正确执行 JUnit 和 TestNG 测试？

Question

Since a while it is possible to configure maven surefire to execute jUnit tests and testNG tests in one build. I won't expand on the reasons why I'm doing that (ok, hint: testNG is our standard framework but some frameworks like jnario require jUnit).

The general way to do it is to configure the surefire plugin like this:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
</plugin>

(taken from here)

This works quite well, jUnit tests and testNG tests get executed.

BUT - now I see that testNG tries to execute the jUnit tests too (and maybe vice-versa) - with no success, of course, because it won't see any of its annotations, but it looks like it re-marks the tests to "passed"... anyway, some reporting tools don't show test fails in jUnit tests anymore unless I comment the second dependency entry.

Is there any better way to configure surefire so that tests from both frameworks are executed ONLY by their test runners?

Answer 1

Had the same problem as you. Zero results in TestNG reports but looks like tests were runned. Finally get this work on two separate executions of maven-surefire-plugin

1. First remove the dependencies section from plugin.
   • otherwise if you use <testNGArtifactName> (and we'll need it) tests will be executed by TestNG anyway and will fail on java.lang.NullPointerException
2. Have some separation of TestNG tests and JUnit tests. In our case NG test class names ends with TestNg
3. skip default execution
4. define two <executions> like below:

<plugin>
    <!-- configured to work with JUnit and TestNg in same project separatelly -->
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>              
    <configuration>
        <!-- whatever you need... -->
        <!-- skip the default execution, we have separate for Ng and for JUnit -->
        <skip>true</skip>
    </configuration>
    <executions>
        <execution>
            <id>TestNg-execution</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <!-- overwrite skip from default config -->
                <skip>false</skip>
                <includes>
                    <include>%regex[.*TestNg.*]</include>
                </includes>
                <!-- used to skip JUnit profider -->
                <junitArtifactName>dev:null</junitArtifactName>
                <!-- to continue on next execution in case of failures here -->
                <testFailureIgnore>true</testFailureIgnore>
            </configuration>
        </execution>
        <execution>
            <id>JUnit-execution</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>

                <skip>false</skip>
                <!-- used to skip TestNg profider -->
                <testNGArtifactName>dev:null</testNGArtifactName>
                <excludes>
                    <exclude>%regex[.*TestNg.*]</exclude>                               
                </excludes>
            </configuration>                    
        </execution>
    </executions>
</plugin>

Tested with:

<junit-version>4.11</junit-version>
<maven-surefire-plugin-version>2.16</maven-surefire-plugin-version>
<org.testng-version>6.8.7</org.testng-version>

This configuration is the marge of ideas from:
Running TestNG-, JUnit3- and JUnit4-tests with Maven Surefire in one run ,
your SUREFIRE bug report ,
surefire plugin configuration

Hope it helps.

Answer 2

I haven't done it myself, but you could try to configure two different executions of the maven-surefire-plugin and configure one of these executions to only run JUnit tests and the other to only run testNG tests. To differentiate between JUnit und testNG tests you should either put the tests in different directories or use a distinguishable name scheme and configure each of the executions with suitable inclusion and exclusion patterns .

In theory this should work :)

Answer 3

不确定此页面底部的运行 TestNG 和 JUnit 测试部分是否在 2013 年存在，但在surefire-testng提供程序中设置junit=false现在似乎可以回答您的问题。