当Maven单元测试失败时，如何使Jenkins构建失败？

Question

I'm using Jenkins, Maven 3.1, and Java 1.6. I have the following Maven job set up in Jenkins with the following goals and options ...

clean install -U -P cloudbees -P qa

below is my pom.xml surefire configuration ...

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.17</version>
        <configuration>
                <reuseForks>true</reuseForks>
                <argLine>-Xmx2048m -XX:MaxPermSize=512M </argLine>
                <skipTests>false</skipTests>
        </configuration>
</plugin>

However, when my unit tests fail, the Jenkins console output still says "BUILD SUCCESS" and the build is marked as "unstable" instead of outright failing. How do I configure things in Jenkins (or Maven if taht's what it turns out to be) so that my build fails (not becomes unstable or passes) if any of the unit tests fail?

Below is what the console output says

17:08:04   MyProjectOrganizationControllerTest.testRecoverFromError » IllegalState Failed to...
17:08:04   MyProjectOrganizationControllerTest.testVerifyDistrictListPopulated » IllegalState
17:08:04   MyProjectOrganizationControllerTest.testUpdateSchool » IllegalState Failed to loa...
17:08:04   MyProjectOrganizationControllerTest.testDeleteSchool » IllegalState Failed to loa...
17:08:04   MyProjectOrganizationControllerTest.testVerifyOrgListPopulatedPrivateSchoolOrgType » IllegalState
17:08:04   MyProjectOrganizationControllerTest.testSubmitMultipleParams » IllegalState Faile...
17:08:04 
17:08:04 Tests run: 155, Failures: 0, Errors: 154, Skipped: 1
17:08:04 
17:08:04 [ERROR] There are test failures.
17:08:04 
17:08:04 Please refer to /scratch/jenkins/workspace/MyProject/MyProject/target/surefire-reports for the individual test results.
17:08:04 [JENKINS] Recording test results
17:08:07 log4j:WARN No appenders could be found for logger (org.apache.commons.beanutils.converters.BooleanConverter).
17:08:07 log4j:WARN Please initialize the log4j system properly.
17:08:14 [INFO] 
17:08:14 [INFO] --- maven-war-plugin:2.4:war (default-war) @ MyProject ---
17:08:15 [INFO] Packaging webapp
17:08:15 [INFO] Assembling webapp [MyProject] in [/scratch/jenkins/workspace/MyProject/MyProject/target/MyProject]
17:08:15 [INFO] Processing war project
17:08:15 [INFO] Copying webapp resources [/scratch/jenkins/workspace/MyProject/MyProject/src/main/webapp]
17:08:15 [INFO] Webapp assembled in [662 msecs]
17:08:15 [INFO] Building war: /scratch/jenkins/workspace/MyProject/MyProject/target/MyProject.war
17:08:20 [INFO] 
17:08:20 [INFO] --- maven-failsafe-plugin:2.17:integration-test (default) @ MyProject ---
17:08:20 [JENKINS] Recording test results
17:08:25 [INFO] 
17:08:25 [INFO] --- maven-failsafe-plugin:2.17:verify (default) @ MyProject ---
17:08:25 [INFO] Failsafe report directory: /scratch/jenkins/workspace/MyProject/MyProject/target/failsafe-reports
17:08:25 [JENKINS] Recording test results[INFO] 
17:08:25 [INFO] --- maven-install-plugin:2.4:install (default-install) @ MyProject ---
17:08:25 
17:08:25 [INFO] Installing /scratch/jenkins/workspace/MyProject/MyProject/target/MyProject.war to /home/jenkins/.m2/repository/org/mainco/subco/MyProject/76.0.0-SNAPSHOT/MyProject-76.0.0-SNAPSHOT.war
17:08:25 [INFO] Installing /scratch/jenkins/workspace/MyProject/MyProject/pom.xml to /home/jenkins/.m2/repository/org/mainco/subco/MyProject/76.0.0-SNAPSHOT/MyProject-76.0.0-SNAPSHOT.pom
17:08:26 Started calculate disk usage of build
17:08:26 Finished Calculation of disk usage of build in 0 seconds
17:08:26 Started calculate disk usage of workspace
17:08:26 Finished Calculation of disk usage of workspace in 0 seconds
17:08:26 [INFO] ------------------------------------------------------------------------
17:08:26 [INFO] BUILD SUCCESS
17:08:26 [INFO] ------------------------------------------------------------------------
17:08:26 [INFO] Total time: 11:00.616s
17:08:26 [INFO] Finished at: Mon Feb 23 17:08:26 UTC 2015
17:08:27 [INFO] Final Memory: 90M/414M
17:08:27 [INFO] ------------------------------------------------------------------------
17:08:27 Waiting for Jenkins to finish collecting data
17:08:28 [JENKINS] Archiving /scratch/jenkins/workspace/MyProject/MyProject/pom.xml to org.mainco.subco/MyProject/76.0.0-SNAPSHOT/MyProject-76.0.0-SNAPSHOT.pom
17:08:28 [JENKINS] Archiving /scratch/jenkins/workspace/MyProject/MyProject/target/MyProject.war to org.mainco.subco/MyProject/76.0.0-  SNAPSHOT/MyProject-76.0.0-SNAPSHOT.war
17:08:31 channel stopped
17:08:31 Started calculate disk usage of build
17:08:31 Finished Calculation of disk usage of build in 0 seconds
17:08:31 Started calculate disk usage of workspace
17:08:31 Finished Calculation of disk usage of workspace in 0 seconds
17:08:31 Finished: UNSTABLE

Answer 1

You can add -Dmaven.test.failure.ignore=false to the MAVEN_OPTS if you click on Advanced button in the Build section of your Jenkins Job.

See Maven Surefire Plugin - surefire:test options for reference.

Answer 2

Use Text Finder plugin. Configure it to look for There are test failures and downgrade the build to FAILED

Answer 3

Another hack that can be useful is to use groovy post-build to check and set test result.

eg this groovy gets build result, appends useful things to build description and sets build result to UNSTABLE in the case that there are no pass or fail but all tests skipped.

def currentBuild = Thread.currentThread().executable
// must be run groovy post-build action AFTER harvest junit xml  if you use junit xml test results
testResult1 = currentBuild.testResultAction

currentBuild.setDescription(currentBuild.getDescription() + "\n pass:"+testResult1.result.passCount.toString()+", fail:"+testResult1.result.failCount.toString()+", skip:"+testResult1.result.skipCount.toString())

// if no pass, no fail all skip then set result to unstable
if (testResult1.result.passCount == 0 && testResult1.result.failCount == 0 && testResult1.result.skipCount > 0) {
   currentBuild.result = hudson.model.Result.UNSTABLE
}

currentBuild.setDescription(currentBuild.getDescription() + "\n" + currentBuild.result.toString())

def ws = manager.build.workspace.getRemote()
myFile = new File(ws + "/VERSION.txt")
desc = myFile.readLines()
currentBuild.setDescription(currentBuild.getDescription() + "\n" + desc)