How to execute cucumber feature file parallel

Question

I have below feature files (Separate feature files) in src/test/resources/feature/ and I would like to run them in parallel. Like: One feature file has to execute in chrome and another one has to execute in firefox as mentioned @Tags name.

Feature: Refund item

@chrome
  Scenario: Jeff returns a faulty microwave
    Given Jeff has bought a microwave for $100
    And he has a receipt
    When he returns the microwave
    Then Jeff should be refunded $100

Feature: Refund Money

@firefox
  Scenario: Jeff returns the money
    Given Jeff has bought a microwave for $100
    And he has a receipt
    When he returns the microwave
    Then Jeff should be refunded $100

Can somebody assist me to achieve this.I'm using cucumber-java 1.2.2 version, and AbstractTestNGCucumberTests using as runner. Also, let me know how can I create a Test Runner dynamically by using feature files and make them run in parallel.

Answer 1

Update: 4.0.0 version is available at maven central repository with bunch of changes. for more details go here.

Update: 2.2.0 version is available at maven central repository.

You can use opensource plugin cucumber-jvm-parallel-plugin which has many advantages over existing solutions. Available at maven repository

   <dependency>
     <groupId>com.github.temyers</groupId>
     <artifactId>cucumber-jvm-parallel-plugin</artifactId>
     <version>2.1.0</version>
   </dependency>

1. First you need to add this plugin with required configuration in your project pom file.

    <plugin> <groupId>com.github.temyers</groupId> <artifactId>cucumber-jvm-parallel-plugin</artifactId> <version>2.1.0</version> <executions> <execution> <id>generateRunners</id> <phase>generate-test-sources</phase> <goals> <goal>generateRunners</goal> </goals> <configuration> <!-- Mandatory --> <!-- comma separated list of package names to scan for glue code --> <glue>foo, bar</glue> <outputDirectory>${project.build.directory}/generated-test-sources/cucumber</outputDirectory> <!-- The directory, which must be in the root of the runtime classpath, containing your feature files. --> <featuresDirectory>src/test/resources/features/</featuresDirectory> <!-- Directory where the cucumber report files shall be written --> <cucumberOutputDir>target/cucumber-parallel</cucumberOutputDir> <!-- comma separated list of output formats json,html,rerun.txt --> <format>json</format> <!-- CucumberOptions.strict property --> <strict>true</strict> <!-- CucumberOptions.monochrome property --> <monochrome>true</monochrome> <!-- The tags to run, maps to CucumberOptions.tags property you can pass ANDed tags like "@tag1","@tag2" and ORed tags like "@tag1,@tag2,@tag3" --> <tags></tags> <!-- If set to true, only feature files containing the required tags shall be generated. --> <filterFeaturesByTags>false</filterFeaturesByTags> <!-- Generate TestNG runners instead of default JUnit ones. --> <useTestNG>false</useTestNG> <!-- The naming scheme to use for the generated test classes. One of 'simple' or 'feature-title' --> <namingScheme>simple</namingScheme> <!-- The class naming pattern to use. Only required/used if naming scheme is 'pattern'.--> <namingPattern>Parallel{c}IT</namingPattern> <!-- One of [SCENARIO, FEATURE]. SCENARIO generates one runner per scenario. FEATURE generates a runner per feature. --> <parallelScheme>SCENARIO</parallelScheme> <!-- This is optional, required only if you want to specify a custom template for the generated sources (this is a relative path) --> <customVmTemplate>src/test/resources/cucumber-custom-runner.vm</customVmTemplate> </configuration> </execution> </executions> </plugin> 

2. Now add below plugin just below above plugin which will invoke runner classes generated by above plugin

     <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19</version> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/*IT.class</include> </includes> </configuration> </plugin> 

3. Above two plugins will do magic for cucumber test running in parallel (provided you machine also have advanced hardware support).

4. Strictly provided <forkCount>n</forkCount> here 'n' is directly proportional to 1) Advanced Hardware support and 2) you available nodes ie registered browser instances to HUB.

5. One major and most important changes is your WebDriver class must be SHARED and you should not implement driver.quit() method, as closing is take care by shutdown hook.

    import cucumber.api.Scenario; import cucumber.api.java.After; import cucumber.api.java.Before; import org.openqa.selenium.OutputType; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebDriverException; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.support.events.EventFiringWebDriver; public class SharedDriver extends EventFiringWebDriver { private static WebDriver REAL_DRIVER = null; private static final Thread CLOSE_THREAD = new Thread() { @Override public void run() { REAL_DRIVER.close(); } }; static { Runtime.getRuntime().addShutdownHook(CLOSE_THREAD); } public SharedDriver() { super(CreateDriver()); } public static WebDriver CreateDriver() { WebDriver webDriver; if (REAL_DRIVER == null) webDriver = new FirefoxDriver(); setWebDriver(webDriver); return webDriver; } public static void setWebDriver(WebDriver webDriver) { this.REAL_DRIVER = webDriver; } public static WebDriver getWebDriver() { return this.REAL_DRIVER; } @Override public void close() { if (Thread.currentThread() != CLOSE_THREAD) { throw new UnsupportedOperationException("You shouldn't close this WebDriver. It's shared and will close when the JVM exits."); } super.close(); } @Before public void deleteAllCookies() { manage().deleteAllCookies(); } @After public void embedScreenshot(Scenario scenario) { try { byte[] screenshot = getScreenshotAs(OutputType.BYTES); scenario.embed(screenshot, "image/png"); } catch (WebDriverException somePlatformsDontSupportScreenshots) { System.err.println(somePlatformsDontSupportScreenshots.getMessage()); } } } 

6. Considering you want to execute more than 50 threads ie same no of browser instances are registered to HUB but Hub will die if it doesn't get enough memory therefore to avoid this critical situation you should start hub with -DPOOL_MAX=512 (or larger) as stated in grid2 documentation .

   Really large (>50 node) Hub installations may need to increase the jetty threads by setting -DPOOL_MAX=512 (or larger) on the java command line.

   java -jar selenium-server-standalone-<version>.jar -role hub -DPOOL_MAX=512

Answer 2

Cucumber does not support parallel execution out of the box. I've tried, but it is not friendly.

1. We have to use maven's capability to invoke it in parallel. Refer link
2. Also there is a github project which uses custom plugin to execute in parallel. Refer cucumber-jvm-parallel-plugin

Answer 3

If all you are expecting is to be able to run multiple features in parallel, then you can try doing the following :

• Duplicate the class AbstractTestNGCucumberTests in your test project and set the attribute parallel=true to the @DataProvider annotated method.

Since the default dataprovider-thread-count from TestNG is 10 and now that you have instructed TestNG to run features in parallel, you should start seeing your feature files get executed in parallel.

But I understand that Cucumber reporting is inherently not thread safe, so your reports may appear garbled.

Answer 4

To take maximum advantage of TestNG you can use Testng's third party extension QAF framework. It supports multiple bdd syntax including gherkin using GherkinFactory . While using BDD with QAF, you can take advantage of each TestNG features, including data-providers, parallel execution configuration in different ways (groups/tests/methods), TestNG listeners.

QAF considers each scenario as TestNG test and Scenario Outline as TestNG data-driven test. As qaf provides driver management and resource management in-built, you don't need to write single line of code for driver management or resource management. All you need to do is create TestNG xml configuration file as per your requirement either to run parallel methods (scenarios) or groups or xml test on one or more browser.

It enables different possible configuration combinations . Below is the xml configuration to address this question which will run scenarios in two browsers and in parallel. You can configure number of threads for each browser as standard TestNG xml configuration as well.

<suite name="AUT Test Automation" verbose="0"  parallel="tests">
<test name="Test-on-chrome">
   <parameter name="scenario.file.loc" value="resources/features" />
   <parameter name="driver.name" value="chromeDriver" />           
   <classes>
      <class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
   </classes>
</test>

<test name="Test-on-FF">
   <parameter name="scenario.file.loc" value="resources/features" />
   <parameter name="driver.name" value="firefoxDriver" />           
   <classes>
      <class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
   </classes>
</test>
</suite>

More over the latest BDDTestFactory2 supports syntax that is derived from QAF BDD, Jbehave and gherkin. It supports meta-data from qaf bdd as tags and examples from gherkin. You can take benefit of inbuilt data-providers to provide test data in XML/JSON/CSV/EXCEL/DB using meta-data in BDD.

Answer 5

I achieved cucumber parallelism using courgette-jvm . It worked out of the box and run parallel test at scenario level

Simply inlclude similar runner class in cucumber. My tests are further using RemoteWebdriver to open multiple instances on selenium grid. Make sure grid is up and running and node is registered to the grid.

import courgette.api.CourgetteOptions;
import courgette.api.CourgetteRunLevel;
import courgette.api.CucumberOptions;
import courgette.api.testng.TestNGCourgette;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
@Test
@CourgetteOptions(
        threads = 10,
        runLevel = CourgetteRunLevel.SCENARIO,
        rerunFailedScenarios = true,
        rerunAttempts = 1,
        showTestOutput = true,
        reportTitle = "Courgette-JVM Example",
        reportTargetDir = "build",
        environmentInfo = "browser=chrome; git_branch=master",
        cucumberOptions = @CucumberOptions(
                features = "src/test/resources/com/test/",
                glue = "com.test.stepdefs",                   
                publish = true,
                plugin = {
                        "pretty",
                        "json:target/cucumber-report/cucumber.json",
                        "html:target/cucumber-report/cucumber.html"}
        ))
class AcceptanceIT extends TestNGCourgette {
}

RemoteWebdriver config is

protected RemoteWebDriver createDriver() throws MalformedURLException , IOException {
                   
                 Properties properties = new Properties();                
                ClassLoader loader = Thread.currentThread().getContextClassLoader();                
                  String hubURL = "http://192.168.1.7:65299/wd/hub";
                  System.setProperty("webdriver.gecko.driver", "/Users/amit/Desktop/amit/projects/misc/geckodriver");
                 
                    FirefoxProfile profile = new FirefoxProfile();
                    DesiredCapabilities capabilities = DesiredCapabilities.firefox();
                    capabilities.setCapability(FirefoxDriver.PROFILE, profile);
                    capabilities.setPlatform(Platform.ANY);    
                    FirefoxOptions options = new FirefoxOptions();
                    options.merge(capabilities);                 
                   driver.set(new RemoteWebDriver(new URL(hubURL),options));
                   return driver.get();
        }

Answer 6

Since v4 of Cucumber you can do this without extensions of plugins.

https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-testng#parallel-execution

Cucumber TestNG supports parallel execution of scenarios. Override the scenarios method to enable parallel execution.

 public class RunCucumberTest extends AbstractTestNGCucumberTests { @Override @DataProvider(parallel = true) public Object[][] scenarios() { return super.scenarios(); } }

Maven Surefire plugin configuration for parallel execution

<plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <configuration> <properties> <property> <name>dataproviderthreadcount</name> <value>${threadcount}</value> </property> </properties> </configuration> </plugin> </plugins>

Where dataproviderthreadcount is the default number of threads to use for data providers when running tests in parallel.