Problems running parallel Cucumber tests from Maven in a Windows batch file

Question

I have a suite of Cucumber Selenium tests in a Maven project and have recently configured the surefire plugin to run my tests in parallel. This works fine by simply executing a command similar to this, where test.runner.directory is a property declared in the pom.xml to hold the value of the file path to my Cucumber test runners:

mvn -Dtest.runner.directory=mytestsrunnerdirectory clean test

Next, I wanted to write a Windows batch file to wrap this call. So this is my attempt at a .bat file, in which the file path to the test runners is passed in as a parameter:

set testrunnerdirectory=%1
echo Running Seleium tests with Maven.
call mvn -Dtest.runner.directory=%testrunnerdirectory% clean test
echo Maven build completed

When executed, this gets as far as successfully running my tests in parallel, but then the script simply hangs. I see all the output from the tests appearing in the window and I observe all tests successfully running and completing, but I never get a "Build Success" message and the script never gets as far as printing my final echo message. It simply hangs indefinitely after the final test completes.

Interestingly, If I set test.runner.directory to a directory with only a single test runner in it, everything works fine (obviously this only launches a single test thread) and the script completes in its entirety. But it seems that the launching of multiple threads somehow throws it and at some point focus cannot be returned to the original command window.

As ever, I would be extremely grateful for any thoughts/advice. Thank you.

Answer 1

I got round this issue by using a plugin in my pom.xml : cucumber-jvm-parallel-plugin

Originally, I had tried to configure surefire to run Test Runners that I had written myself, as per the snippet below: -

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20.1</version>
            <configuration>
                <forkCount>5</forkCount>
                <reuseForks>false</reuseForks>
                <testFailureIgnore>true</testFailureIgnore>
                <includes>
                    <include>*TestRunner.java</include>
                </includes>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.apache.maven.surefire</groupId>
                    <artifactId>surefire-junit47</artifactId>
                    <version>2.20.1</version>
                </dependency>
            </dependencies>
        </plugin>

As per my original question, this ran my tests OK, but caused my batch script to hang and never complete. The new approach using cucumber-jvm-parallel-plugin is shown in the snippets below:

        <plugin>
            <groupId>com.github.temyers</groupId>
            <artifactId>cucumber-jvm-parallel-plugin</artifactId>
            <version>1.0.1</version>
            <executions>
                <execution>
                    <id>generateRunners</id>
                    <phase>validate</phase>
                    <goals>
                        <goal>generateRunners</goal>
                    </goals>
                    <configuration>
                        <glue>mystepdefinitionspackage</glue>
                        <featuresDirectory>myfeatures</featuresDirectory>
                        <cucumberOutputDir>myoutput</cucumberOutputDir>
                        <format>json,html</format>
                        <tags>"@MyTag"</tags>
                    </configuration>
                </execution>
            </executions>
        </plugin>

This plugin generates Test Runners on the fly, and surefire just needs to be configured to pick up those generated Test Runners:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20.1</version>
            <configuration>
                <forkCount>10</forkCount>
                <reuseForks>true</reuseForks>
                <includes>
                    <include>**/Parallel*IT.class</include>
                </includes>
            </configuration>
        </plugin>

This worked brilliantly. My tests run in parallel and the script completes gracefully without hanging. Thank you for your interest in my question.