How do I run my cucumber-jvm features in parallel? (with maven using cucumber-jvm-parallel-plugin)

Question

I am trying to incorporate a maven plugin cucumber-jvm-parallel-plugin into my Cucumber-JVM code and am running into some problems... I think I have configured my pom.xml properly, but my cucumber features are still getting ran one by one instead of in parallel .

How do I configure my pom.xml to make my cucumber features run in parallel? I have some feature files that block execution and I don't want to have to wait for them to finish in order to start testing the other features.

I followed two tutorials, and can't find where I am going wrong, or if this is expected:

• https://opencredo.com/test-automation-concepts-parallel-test-execution/
• http://automationrhapsody.com/running-cucumber-tests-in-parallel/

Structure

Here's my program's structure:

├── features/
│   └── api/
│       ├── 006-Email.feature
│       └── 999-Login.feature
├── results/
│   └── api-json.json
└── src/
    └── java/
        ├── pom.xml
        ├── src/
        │   └── test/
        │       └── java/
        │           └── com/
        │               └── mycompany/
        │                   └── commonapps/
        │                       └── queuemanager/
        │                           ├── Globals.java
        │                           ├── GmailHelper.java
        │                           ├── HttpDeleteWithBody.java
        │                           ├── JsonHelper.java
        │                           ├── RequestHelper.java
        │                           ├── RunApiTest.java
        │                           ├── ThreadedSteps.java
        │                           └── UserHelper.java
        └── steps/
            └── com/
                └── mycompany/
                    └── commonapps/
                        └── queuemanager/
                            ├── GivenSteps.java
                            ├── Setup.java
                            ├── ThenSteps.java
                            └── WhenSteps.java

Maven Pom Logic

The way I configured my pom is as follows:

1. I copy ../../features/api/ to my build path using Maven Resources Plugin

    <build> <resources> <resource> <directory>../../features/api/</directory> </resource> </resources> </build> 

2. I tell cucumber-jvm-parallel-plugin that my features are now located at target/classes/active/ , because I think ../../features/api/ was copied to target/ , so target/active/ should contain my .feature files.. I think.

     <plugin> <groupId>com.github.temyers</groupId> <artifactId>cucumber-jvm-parallel-plugin</artifactId> ... <configuration> ... <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory> ... </configuration> </plugin> 

3. Then, after I run maven test , all .feature files are then compiled into target/test-classes/Parallel**IT.class . One per feature file.

4. So I tell maven-surefire-plugin to run all <include>**/Parallel*IT.class</include> files in parallel, which are the feature files created by cucumber-jvm-parallel-plugin.

     <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <executions> <execution> <id>acceptance-test</id> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/Parallel*IT.class</include> </includes> </configuration> </execution> </executions> </plugin> 

Full pom.xml

And my pom.xml is:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.mycompany.commonapps</groupId>
   <artifactId>e2e-server</artifactId>
   <version>0.0.1-SNAPSHOT</version>
   <dependencies>
     ...
   </dependencies>
   <build>
<!-- COPY RESOURCES TO TARGET/ -->
      <resources>
    <!-- copies features to target: "${project.build.directory}/classes/active/" -->
         <resource>
            <directory>../../features/api/</directory> 
         </resource>
      </resources>
<!-- RUN EACH CUCUMBER FEATURE FILE AS A FORK -->
      <plugins>
    <plugin>
      <groupId>com.github.temyers</groupId>
      <artifactId>cucumber-jvm-parallel-plugin</artifactId>
      <version>1.2.1</version>
      <executions>
        <execution>
          <id>generateRunners</id>
          <phase>generate-test-sources</phase>
          <goals>
        <goal>generateRunners</goal>
          </goals>
          <configuration>
          <glue>com.mycompany.commonapps.queuemanager</glue>
           <outputDirectory>${project.build.directory}/generated-test-sources</outputDirectory>
           <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory>
           <cucumberOutputDir>${project.build.directory}</cucumberOutputDir>
           <format>json</format>
           <!--<strict>true</strict>
           <monochrome>true</monochrome> 
           <useTestNG>false</useTestNG>
           <namingScheme>simple</namingScheme>-->
           <parallelScheme>FEATURE</parallelScheme>
          </configuration>
        </execution>
      </executions>
    </plugin>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
<!-- RUN ALL CLASSES CREATED BY cucumber-jvm-parallel-plugin IN PARALLEL -->
        <executions>
         <execution>
             <id>acceptance-test</id>
             <phase>integration-test</phase>
             <goals>
                 <goal>test</goal>
             </goals>
             <configuration>
                 <forkCount>5</forkCount>
                 <reuseForks>true</reuseForks>
                 <includes>
                    <include>**/Parallel*IT.class</include>
                 </includes>
             </configuration>
        </execution>
        </executions>
         </plugin>
<!-- ADD ../../STEPS/ TO THE SOURCE -->
         <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.7</version>
            <executions>
               <execution>
                  <id>add-source</id>
                  <phase>generate-sources</phase>
                  <goals>
                     <goal>add-source</goal>
                  </goals>
                  <configuration>
                     <sources>
                        <source>steps/</source>
                        <source>src/test/java/</source>
                     </sources>
                  </configuration>
               </execution>
            </executions>
         </plugin>
      </plugins>
   </build>
</project>

Cucumber Runner

And my cucumber running class looks like this:

@RunWith(Cucumber.class)
@CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"})
public class RunApiTest {
}

Output

When I run mvn test , I get the following:

...
[INFO] ------------------------------------------------------------------------
[INFO] Building e2e  Server Tests 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2
[INFO] 
[INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @ e2e--server ---
[INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/steps added.
[INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java added.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ e2e--server ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 7 resources
[INFO] Copying 2 resources
[INFO] Copying 7 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ e2e--server ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 12 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/classes
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations.
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- cucumber-jvm-parallel-plugin:1.2.1:generateRunners (generateRunners) @ e2e--server ---
[INFO] Adding /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/generated-test-sources to test-compile source root
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ e2e--server ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ e2e--server ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 10 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/test-classes
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations.
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ e2e--server ---
[INFO] Surefire report directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.mycompany.commonapps.queuemanager.RunApiTest
....
*Tests run one by one properly*
...
Tests run: 31, Failures: 2, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:40 min
[INFO] Finished at: 2016-11-10T15:49:11-08:00
[INFO] Final Memory: 20M/261M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project e2e--server: There are test failures.
[ERROR] 
[ERROR] Please refer to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Does anyone see what I did wrong?

---

More Debugging Info

• cucumber-jvm-parallel-plugin's outputDirectory is getting created correctly and contains a bunch of Parallel**IT.class files
• cucumber-jvm-parallel-plugin's featuresDirectory is getting accessed correctly .
• cucumber-jvm-parallel-plugin's cucumberOutputDir is NOT getting created at all after a build.

Answer 1

I finally got it to work! Here's what I had to do to make my tests run in parallel:

1. I move some fields in the maven-surefire-plugin execution tag to the configuration tag. I moved this section:

     <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/*IT.class</include> </includes> 

   • So that my maven-surefire-plugin now looks like this:

     <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19.1</version> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/*IT.class</include> </includes> </configuration> </plugin> 

2. Since cucumber-jvm-parallel-plugin is now running the show, I had to remove my cucumber runner class . I commented it out like this:

    /*
    @RunWith(Cucumber.class)
    @CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"})
    */
    public class RunApiTest {
    }

And that's it. Since Parallel**IT.class where getting created, it was maven-surefire-plugin that was not working right.