No Failure with IT tests when Exception is thrown Arquillian TestNG maven

Question

I make a simple test with Arquillian and testNG and maven

@Test(enabled=true)
@OperateOnDeployment("test")
public void testError() throws Exception {
    throw new Exception();
}

in theory the test will be in error and following tests will not be executed but nothing append, all tests are OK in results, so i can't really test my functionalities because even if an exception is thrown no error append. It is not normal.

i recreate a simple webapp with following pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>sample.spnego</groupId>
<artifactId>sample-app</artifactId>
<packaging>war</packaging>
<version>1.0</version>
<name>sample-app</name>

<dependencyManagement>
    <dependencies>
        <!-- Define the version of JBoss' Java EE 6 APIs we want to import. Any 
            dependencies from org.jboss.spec will have their version defined by this 
            BOM -->
        <dependency>
            <groupId>org.jboss.spec</groupId>
            <artifactId>jboss-javaee-6.0</artifactId>
            <version>3.0.3.Final</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian</groupId>
            <artifactId>arquillian-bom</artifactId>
            <!-- <version>1.1.8.Final</version> -->
            <version>1.1.12.Final</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>




<dependencies>
    <dependency>
        <groupId>commons-io</groupId>
        <artifactId>commons-io</artifactId>
        <version>2.5</version>
    </dependency>
    <dependency>
        <groupId>org.jboss.spec</groupId>
        <artifactId>jboss-javaee-6.0</artifactId>
        <version>1.0.0.Final</version>
        <type>pom</type>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>javax.enterprise</groupId>
        <artifactId>cdi-api</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.jboss.spec.javax.annotation</groupId>
        <artifactId>jboss-annotations-api_1.1_spec</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.jboss.spec.javax.ejb</groupId>
        <artifactId>jboss-ejb-api_3.1_spec</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.10</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.jboss.arquillian.testng</groupId>
        <artifactId>arquillian-testng-container</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.jboss.arquillian.protocol</groupId>
        <artifactId>arquillian-protocol-servlet</artifactId>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.eu.ingwar.tools</groupId>
        <artifactId>arquillian-suite-extension</artifactId>
        <version>1.1.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.6.4</version>
        <scope>provided</scope>
    </dependency>


    <dependency>
        <groupId>org.apache.nutch</groupId>
        <artifactId>nutch</artifactId>
        <version>2.3.1</version>
    </dependency>
    <dependency>
        <groupId>org.wildfly.arquillian</groupId>
        <artifactId>wildfly-arquillian-container-managed</artifactId>
        <version>2.0.2.Final</version>
        <scope>test</scope>
    </dependency>
</dependencies>



<profiles>
    <profile>
        <id>integration</id>
        <build>
            <plugins>
                <!-- <plugin> -->
                <!-- <groupId>org.apache.maven.plugins</groupId> -->
                <!-- <artifactId>maven-surefire-plugin</artifactId> -->
                <!-- <version>2.19.1</version> -->
                <!-- </plugin> -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <version>2.19.1</version>

                    <configuration>
                        <systemPropertyVariables>
                            <jboss.server.log.dir>C:/dev/servers/wildfly-9.0.2.Final/standalone/log</jboss.server.log.dir>
                            <arquillian.launch>jbossas-managed</arquillian.launch>
                            <jbossas.startup.timeout>240</jbossas.startup.timeout>

                        </systemPropertyVariables>
                        <includes>
                            <include>*.java</include>
                        </includes>
                        <!-- <groups>org.jboss.arquillian.testng.Arquillian</groups> -->
                    </configuration>
                    <executions>
                        <execution>
                            <goals>
                                <goal>integration-test</goal>
                                <goal>verify</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>

    </profile>
</profiles>

then following ArquillianDeploymentHelper :

@ArquillianSuiteDeployment
public class ArquillianDeploymentHelper {

//private static final Logger LOGGER = Logger.getLogger(ArquillianDeploymentHelper.class);

private static String WEBAPP_SRC = "src/main/webapp";
private static String POM_FILE = "pom.xml";
private static String ARCHIVE_NAME = "test.war";

@Deployment(name = "test")
public static Archive<?> generateDefaultDeployment() {

    // Generate the default WAR used by all *IT tests using @OperateOnDeployment("AofWeb") annotation
    //LOGGER.info("Generating " + ARCHIVE_NAME + " archive ...");

    PomEquippedResolveStage pom = Maven.resolver().loadPomFromFile(POM_FILE);
    ScopeType[] scopes = {ScopeType.COMPILE, ScopeType.IMPORT, ScopeType.TEST}; // no SYSTEM and no PROVIDED
    File[] libs = pom.importDependencies(scopes).resolve().using(TransitiveStrategy.INSTANCE).asFile();

    WebArchive archive =  ShrinkWrap.create(WebArchive.class, ARCHIVE_NAME)

            .addAsWebInfResource(new File(WEBAPP_SRC, "WEB-INF/beans.xml"));
    archive.merge(ShrinkWrap.create(GenericArchive.class).as(ExplodedImporter.class)  
            .importDirectory(WEBAPP_SRC).as(GenericArchive.class),  
            "/", Filters.include(".*\\.(?i)(jpg|js|png|gif|bmp|html|xhtml|css)$"));  

    // No need to log the content anymore, the archive is kept in target directory 
    // "deploymentExportPath" variable in arquillian.xml
//  LOGGER.info(archive.toString(true));

    return archive;
}

}

and arquillian.xml:

<?xml version="1.0" encoding="UTF-8"?>
<arquillian xmlns="http://jboss.org/schema/arquillian"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://jboss.org/schema/arquillian      http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

<!-- Force the use of the Servlet 3.0 protocol with all containers, as it is the most mature -->
<defaultProtocol type="Servlet 3.0" />

<!-- Keep generated archive for inspection
 -->
<engine>
       <property name="deploymentExportPath">target/</property>
</engine>



<!-- Local JBoss/WildFly instance -->
<container qualifier="jbossas-managed" default="true">
    <configuration>
        <property name="jbossHome">${env.JBOSS_HOME}</property>
        <property name="serverConfig">standalone-real-continuous-integration.xml</property>
         <!-- true : server must be started by the user, false : arquillian will start it itself -->
        <property name="allowConnectingToRunningServer">true</property>
    </configuration>
</container>

<!-- Remote JBoss/WildFly instance -->
<container qualifier="jbossas-remote" default="false">
    <configuration>
        <property name="managementAddress">remoteHost</property>
        <property name="managementPort">9999</property>
    </configuration>
</container>

wildfly version 9.0.2

NOTE : i test Arquillian tutorial it is same version of bom and test in jar instead of war and the exception produce a test failure =>OK

above configuration : exception not produce a failut => NOK

Any idea ?

EDIT : i reproduce the problem with adding in test working :

        <dependency>
        <groupId>org.wildfly.arquillian</groupId>
        <artifactId>wildfly-arquillian-container-managed</artifactId>
            <version>2.1.0.Alpha1</version>
        <scope>test</scope>
    </dependency>

and using Arquillian.xml

instead :

  <dependency> 
 <groupId>org.jboss.arquillian.container</groupId>
 <artifactId>arquillian-weld-ee-embedded-1.1</artifactId> 
 <version>1.0.0.CR9</version> 
 <scope>test</scope> 
 </dependency> 
<dependency> 
<groupId>org.jboss.weld</groupId> 
<artifactId>weld-core</artifactId> 
<version>2.3.5.Final</version> 
<scope>test</scope> 
</dependency> 

Answer 1

最后，如果我降级到6.9.10，它似乎是最新的tesNG版本6.10，它可以正常工作！

Answer 2

Maybe this will help you: https://stackoverflow.com/a/32066322/4471199

surefire-maven-plugin does not recognize tests that ends with *IT. You should change it's name to *Test or use other technique