使用Failsafe + Cucumber Parallel插件时无法使用@BeforeClass @AfterClass等Junit或TestNG注释

Question

I am trying to do Parallel testing using Selenium and Cucumber.So we have around 130 test scenarios which takes around 1.5 hours to run. Now By running all these tests in Parallel I can reduce the time.

I am using following 2 plugins:

 <plugin>
                <groupId>com.github.temyers</groupId>
                <artifactId>cucumber-jvm-parallel-plugin</artifactId>
                <version>5.0.0</version>
                <executions>
                    <execution>
                        <id>generateRunners</id>
                        <phase>generate-test-sources</phase>
                        <goals>
                            <goal>generateRunners</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>runner</outputDirectory>
                            <glue>
                                <package>${PackageName}</package>
                            </glue>
                            <featuresDirectory>${FeaturesDirectory}</featuresDirectory>
                            <cucumberOutputDir>target/cucumber-parallel</cucumberOutputDir>
                            <format>json</format>
                            <plugins>
                                <plugin>
                                    <name>json</name>
                                </plugin>
                            </plugins>
                            <useTestNG>true</useTestNG>

                                                       <parallelScheme>SCENARIO</parallelScheme>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>3.0.0-M3</version>
                <executions>
                                        <execution>
                        <id>acceptance-test</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>

                        </goals>

                        <configuration>
                            <forkCount>3</forkCount>
                            <reuseForks>true</reuseForks>
                            <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
                            <includes>
                                <include>**/Parallel*IT.class</include>
                            </includes>
                            <properties>
                                <property>
                                    <name>junit</name>
                                    <value>false</value>
                                </property>
                            </properties>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

So the idea is, suppose I give Thread count or Fork count 3, it will invoke 3 browsers, and it will run the tests. With CUcumber JVM Parallel Plug in I have created 131 runner for all the scenarios. With @Before annotation from Cucumber, I can invoke the browser once, and for all the next instances it will use the same browser instance (Thus saving the time to open and close the browsers again and again.)

Now if we can use @BeforeClass or @AfterClass, this would have been much simpler solution. But as we can not. I had to use @Before, thus calling the same method again and again with each scenario.

import cucumber.api.java.Before;


public class stepdefs(){
static Browser browser;

    public static Scenario scenario;
    static TestConfiguration configuration = TestConfiguration.getConfiguration();
    @Before
    public void before(Scenario scenario) {

        if (browser == null) {

            if (System.getProperty("seleniumRemoteWebDriver") == null) {

                WebTestSettings webTestSettings = configuration.getWebTestSettings();

                browser = Browser.LOAD_LOCAL_BROWSER(webTestSettings.externalBrowser, webTestSettings.driverPath);

            } else {
                if (System.getProperty("version") == null) {
                    browser = new RemoteBrowser(System.getProperty("browser"), System.getProperty("seleniumRemoteWebDriver"), System.getProperty("platform"));

                } else {
                    browser = new RemoteBrowser(System.getProperty("browser"), System.getProperty("seleniumRemoteWebDriver"), System.getProperty("platform"), System.getProperty("version"));
                }
            }

            TestContext.getContext().setBrowser(browser);


        }

        Browser browser = TestContext.getContext().getBrowser();
        this.scenario = scenario;
        browser.driver.manage().window().maximize();
           }}

So this worked for @Before, but this will not work for @After. What I want to achieve here is to close all the browser instances after the test is done. But How do I know which 1 is the final instance? Or how do I close all the instances? Is there a way to use @AfterClass or @BeforeCLass or use SetupTeardown with JUnit/TestNG + Maven-Failsafe-plugin?

I have tried using Junit and TestNG both. I used to use a Runner file which extends to the SetupTearDown class for the non-parallel execution. But obviously can not use that as the runner classes are getting generated in runtime. Tried using pre-integration , post-integration phase as well. Which did not run. May be I am not using that phase correctly.

Answer 1

Lets move step by step. First we shall use Cucumber V starting from 4.0 onward as it supports parallel execution and we shall add below dependencies in order to implement it.

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>4.2.3</version>
</dependency>

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>4.2.3</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>datatable</artifactId>
    <version>1.1.12</version>
</dependency>

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-testng</artifactId>
    <version>4.2.3</version>
</dependency>

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-picocontainer</artifactId>
    <version>4.2.3</version>
    <scope>test</scope>
</dependency>  

Then we shall start using maven-surefire plugin in order to actually run tests in parallel using parallel & threadCount properties as mentioned below -

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${maven-surefire.plugin.version}</version>
    <configuration>
        <parallel>methods</parallel>
        <threadCount>1</threadCount>
        <reuserForks>false</reuserForks>
        <testErrorIgnore>true</testErrorIgnore>   
        <testFailureIgnore>true</testFailureIgnore>
        <includes>
            <include>**/*RunCukeTest.java</include>
        </includes>
    </configuration>
</plugin>

Depending on implementation, it would run test cases in different browser & close all. Now if you want to do something specific only once before or after the execution, you shall use @BeforeClass or @AfterClass in RunCuke Java file. Things are Thread Safe.

Answer 2

Looking into your existing code it's not thread safe and will not allow you parallel execution because of static Browser . You should try using QAF - Pure TestNG implementation for BDD/Gherkin Which provides gherkin implementation that allows you to use all TestNG listener as well as have additional listeners for driver and element. Furthermore driver management taken care by qaf so you may not need to write additional code to manage driver or other test automation specific needs. qaf will take care of close all the browser instances after the test and will also allow you scenario level parallel execution.