Selenium and Parallelized JUnit - WebDriver instances

Question

The setup

So, basically i am trying to achieve Selenium tests that run in parallel using JUnit.

For that i have found this JUnit runner . It works really well, i like it alot.

However, i am running into problems regarding the handling of WebDriver instances.

What i want

Each WebDriver element should be created once for every class before @Test methods are executed.

Logically, i could use the classes constructor for this. Actually this is quite the requirement for my tests because i need to make use of the @Parameters so that i can create the WebDriver instance accordingly (Chrome,FF,IE ...).

The problem

The problem is that i want the WebDriver instance to be cleared ( driver.quit() ) after a class is done and not after each @Test method is done. But i cannot use @AfterClass because i cannot make WebDriver a static member, since each class instance has to use its own (otherwise tests would try to run in the same browser).

A possible solution

I have found a possible suggestion here by Mrunal Gosar . Following his advise i have changed WebDriver to be a static ThreadLocal<WebDriver> instead and then i create instances of it in each constructor using

 // in the classes constructor

driver = new ThreadLocal<WebDriver>() {
           @Override
           protected WebDriver initialValue() {
                 return new FirefoxDriver(); /
           }
};

Needles to say that i replaced every driver.whatever call with driver.get().whatever in my code.

Now, to address the ultimate purpose of this i also wrote a @AfterClass method that would call driver.get().quit(); which is now accepted by the compiler, as the variable is static.

Testing this however leads to unexpected behavior. I have a Selenium Grid setup with 2 nodes running on a remote machine. I had this setup running as expected before, but now browsers are spammed all over and tests fail. (While 2 browsers should be running instead 8+ are opened)

The thread i linked suggesting this solution had someone commenting that it might be a bad idea to manually handle threads if already using a framework like JUnit.

My Question

What is the right design to do this?

I could only think of

1. Make what is suggested here work
2. Write a single @Test annotated method that executes all other methods and then use @After to achieve the same as @AfterClass
3. Save the constructor parameter in a member variable and deal with the fact that i have to create a browser before each @Test annotated method is executed (Using @Before to create the WebDriver instance and @After to close the session)

I don't quite know if option 3 runs into possible problems though. If i close the session after each method, then the grid-Server might actually open a new session with an entirely new class on this node before this one has finished the previous ones. While the tests are independent of each other, i still feel like this is potential danger.

Is anyone on here actively using a multithreaded Selenium test suit and can guide me what is proper design?

Answer 1

In general I agree that:

it might be a bad idea to manually handle threads if already using a framework like JUnit

But, looking at Parallelized runner you mentioned and internal implementation of @Parametrized in junit 4.12 it is possible.

Each test case is scheduled for execution. By default junit executes test cases in single thread. Parallelized extends Parametrized in that way that single threaded test scheduler is replaced with multi-thread scheduler so, to understand how this affects way Parametrized test cases are run we have to look inside JUnit Parametrized sources:

https://github.com/junit-team/junit/blob/r4.12/src/main/java/org/junit/runners/Parameterized.java#L303

Looks like:

1. @Parametrized test case is split to group of TestWithParameters for every test parameter
2. for every TestWithParameters instance of Runner is created and scheduled for execution (in this case Runner instance is specialized BlockJUnit4ClassRunnerWithParameters )

In effect, every @Parametrized test case generates group of test instances to run (single instance for every parameter) and every instance is scheduled independently so in our case ( Parallelized and @Parametrized test with WebDriver instances as parameters) multiple independent tests will be executed in dedicated threads for every WebDriver type. And this is important because allows us to store specific WebDriver instance in scope of the current thread.

Please remember this behavior relies on internal implementation details of junit 4.12 and may change (for example see comments in RunnerScheduler ).

Take I look at example below. It relies on mentioned JUnit behavior and uses ThreadLocal to store WebDriver instances shared between test in the same cases groups. Only trick with ThreadLocal is to initialize it only once (in @Before) and destroy every created instance (in @AfterClass).

package example.junit;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;

import org.apache.commons.lang3.StringUtils;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

/**
 * Parallel Selenium WebDriver example for http://stackoverflow.com/questions/30353996/selenium-and-parallelized-junit-webdriver-instances
 * Parallelized class is like http://hwellmann.blogspot.de/2009/12/running-parameterized-junit-tests-in.html
 */
@RunWith(Parallelized.class)
public class ParallelSeleniumTest {

    /** Available driver types */
    enum WebDriverType {
        CHROME,
        FIREFOX
    }

    /** Create WebDriver instances for specified type */
    static class WebDriverFactory {
        static WebDriver create(WebDriverType type) {
            WebDriver driver;
            switch (type) {
            case FIREFOX:
                driver = new FirefoxDriver();
                break;
            case CHROME:
                driver = new ChromeDriver();
                break;
            default:
                throw new IllegalStateException();
            }
            log(driver, "created");
            return driver;
        }
    }

    // for description how to user Parametrized
    // see: https://github.com/junit-team/junit/wiki/Parameterized-tests
    @Parameterized.Parameter
    public WebDriverType currentDriverType;

    // test case naming requires junit 4.11
    @Parameterized.Parameters(name= "{0}")
    public static Collection<Object[]> driverTypes() {
        return Arrays.asList(new Object[][] {
                { WebDriverType.CHROME },
                { WebDriverType.FIREFOX }
            });
    }

    private static ThreadLocal<WebDriver> currentDriver = new ThreadLocal<WebDriver>();
    private static List<WebDriver> driversToCleanup = Collections.synchronizedList(new ArrayList<WebDriver>());

    @BeforeClass
    public static void initChromeVariables() {
        System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
    }

    @Before
    public void driverInit() {
        if (currentDriver.get()==null) {
            WebDriver driver = WebDriverFactory.create(currentDriverType);
            driversToCleanup.add(driver);
            currentDriver.set(driver);
        }
    }

    private WebDriver getDriver() {
        return currentDriver.get();
    }

    @Test
    public void searchForChromeDriver() throws InterruptedException {
        openAndSearch(getDriver(), "chromedriver");
    }

    @Test
    public void searchForJunit() throws InterruptedException {
        openAndSearch(getDriver(), "junit");
    }

    @Test
    public void searchForStackoverflow() throws InterruptedException {
        openAndSearch(getDriver(), "stackoverflow");
    }

    private void openAndSearch(WebDriver driver, String phraseToSearch) throws InterruptedException {
        log(driver, "search for: "+phraseToSearch);
        driver.get("http://www.google.com");
        WebElement searchBox = driver.findElement(By.name("q"));
        searchBox.sendKeys(phraseToSearch);
        searchBox.submit();
        Thread.sleep(3000);
    }

    @AfterClass
    public static void driverCleanup() {
        Iterator<WebDriver> iterator = driversToCleanup.iterator();
        while (iterator.hasNext()) {
            WebDriver driver = iterator.next();
            log(driver, "about to quit");
            driver.quit();
            iterator.remove();
        }
    }

    private static void log(WebDriver driver, String message) {
        String driverShortName = StringUtils.substringAfterLast(driver.getClass().getName(), ".");
        System.out.println(String.format("%15s, %15s: %s", Thread.currentThread().getName(), driverShortName, message));
    }

}

It will open two browsers and execute three test cases in every browser window concurrently .

Console will print something like:

pool-1-thread-1,    ChromeDriver: created
pool-1-thread-1,    ChromeDriver: search for: stackoverflow
pool-1-thread-2,   FirefoxDriver: created
pool-1-thread-2,   FirefoxDriver: search for: stackoverflow
pool-1-thread-1,    ChromeDriver: search for: junit
pool-1-thread-2,   FirefoxDriver: search for: junit
pool-1-thread-1,    ChromeDriver: search for: chromedriver
pool-1-thread-2,   FirefoxDriver: search for: chromedriver
           main,    ChromeDriver: about to quit
           main,   FirefoxDriver: about to quit

You can see that drivers are created once for every worker thread and destroyed at the end.

To summarize, we need something like @BeforeParameter and @AfterParameter in context of execution thread and quick search shows that such idea is already registered as issue in Junit