testng-failed.xml is rerunning all tests and not only the failed ones

Question

First, I execute:

- mvn clean test -Dsurefire.suiteXmlFiles=src/test/resources/testing.xml

Then, it generates as expected the testng-failed.xml file

Then I try to rerun with:

- mvn test -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml

I have 3 tests in this example, and in the 2nd run I get below info:

Failed tests:

PACKAGE_INFO..

• Run 1: FAILED_TEST_NAME
• Run 2: FAILED_TEST_NAME

Tests run: 3, Failures: 1, Errors: 0, Skipped: 0

And the biggest problem is that it is really executing all 3 tests agains, and I just want to re-execute the failed one.

Answer 1

Recently I've faced with the same problem. My test infra was built on Java 1.8 + TestNG 6.9.10 + surefire-plugin. And I've also faced the same problem you've described.

That's not your fault if testng-failed.xml contains all tests from your suite. That problem is described here and is an issue incident to TestNG version 6.9.10.

So the only one working solution for now is to downgrade TestNG version in your pom.xml file(if you use Maven) to 6.9.9 where there's no such issue. Or, you can try to use the latest TestNG snapshot version where the issue had already been fixed.

Answer 2

The bug has already been fixed in 6.9.11. You need to upgrade to that version.

https://github.com/cbeust/testng/blob/master/CHANGES.txt#L290