How to use execution pointcut with third-party maven dependency?

Question

I am trying to develop an AspectJ aspect that will automatically swallow any ElementNotVisibleException or StaleElementReferenceException s (subclasses of RuntimeException ) thrown by Selenium-Java's WebDriver instances (including multiple subclasses of WebDriver -- ChromeDriver , FirefoxDriver , etc.)

Basically, the standard suggested solution in a non-AOP context for handling ENVE and SERE exceptions with Selenium is to simply try again. And again. And again, if necessary.

Something like this would work in a functional paradigm:

public void tryWhileStale(Runnable r)
{
        int n = 0;
        while(n < 5)
        {
            try
            {
                r.run();
                break;
            }
            catch(StaleElementReferenceException | ElementNotVisibleException e){}
            n++;
            Thread.sleep(2000);
        }
        throw new RuntimeException("Timed out retrying");
}

Then, when using WebDriver later:

tryWhileStale(() -> driver.findElement(By.xpath(...)).click());

However, this adds quite a bit of extra typing (as well as the very real possibility of accidentally forgetting the tryWhileStale() wrapper) that I'd like to avoid.

I don't want to download a copy of selenium-java, edit the source and rebuild, because I'm pulling Selenium directly from the public Maven repositories.

I hoped that AspectJ would be able to figure out how to do this, so I did some research and realized I need an around advice with an execution pointcut. If I use call instead of execution , it will trigger successfully, but it won't swallow the exception. The reason for this is mysterious to me, since it seems from my code flow in the aspect I've written would catch anything thrown in the proceed() call.

But an execution() pointcut won't work, either! This is because AspectJ is weaving my classes, but is not weaving Selenium-Java, even though I have a weaveDependency in my pom.xml! The kicker is that call() works only if your classes are woven, while execution() works only if the class you're calling is woven. Obviously, either one works if you have both your classes and the third-party classes woven.

Is there any way to do this without abandoning AOP or Maven entirely? Below is my code that should purportedly work if I were able to weave selenium-java:

@Aspect
class MyAspect {

    @Around("execution (WebElement *.findElement(By))")
    public Object around(ProceedingJoinPoint pjp) 
    {
        Object f = null;
        int n = 0;
        do
        {
            try
            {
                System.err.println("Before " + this.toString());
                f = pjp.proceed();
                System.err.println("After " + this.toString());
                return f;
            }
            catch(Throwable t)
            {
                try { Thread.sleep(5000); } catch(InterruptedException ie) { break; }
                System.err.println("Waiting 5 seconds because of " + t.getClass().getSimpleName());
            }
            n++;
        } while(n < 5);
        System.err.println("Gave up waiting");
        return null;
    }
}

Answer 1

I was a bit curious and set up a sample project with an aspect intercepting call(WebElement WebDriver+.findElement(*)) (similar to your approach) and also call(void WebElement+.click()) . I used a sample page with an inline frame (iframe) from W3schools in order to emulate a few WebDriver exceptions such as NoSuchElementException and StaleElementReferenceException . This is easy if you just switch the focus from the main frame to an iframe and try to access an element from the former (or vice versa).

My sample aspect does not wait for n seconds, but iterates over the main and all iframes in order to re-issue the original call in that context. It should be very easy to adapt the sample code to your needs.

Oh BTW, I used native AspectJ syntax, not the annotation-based one. I hope you do not mind, I find the native syntax more expressive and elegant.

Maven POM to build the project:

The pom.xml contains a few extra plugins for

• building a single, executeable JAR ( one-jar ) containing the AspectJ runtime and all other dependencies. Very convenient for distributing and running your compiled program.
• I also included exec-maven so as to enable you to easily run your program via mvn clean compile exec:java .

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>de.scrum-master.stackoverflow</groupId>
    <artifactId>selenium-aspectj-retry</artifactId>
    <version>1.0-SNAPSHOT</version>

    <name>Selenium auto-retry via AspectJ</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.source-target.version>1.8</java.source-target.version>
        <aspectj.version>1.8.7</aspectj.version>
        <main-class>de.scrum_master.app.Application</main-class>
    </properties>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.3</version>
                    <configuration>
                        <source>${java.source-target.version}</source>
                        <target>${java.source-target.version}</target>
                        <!-- IMPORTANT -->
                        <useIncrementalCompilation>false</useIncrementalCompilation>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>aspectj-maven-plugin</artifactId>
                    <version>1.8</version>
                    <configuration>
                        <!--<showWeaveInfo>true</showWeaveInfo> -->
                        <source>${java.source-target.version}</source>
                        <target>${java.source-target.version}</target>
                        <Xlint>ignore</Xlint>
                        <complianceLevel>${java.source-target.version}</complianceLevel>
                        <encoding>${project.build.sourceEncoding}</encoding>
                        <!--<verbose>true</verbose> -->
                        <!--<warn>constructorName,packageDefaultMethod,deprecation,maskedCatchBlocks,unusedLocals,unusedArguments,unusedImport</warn> -->
                    </configuration>
                    <executions>
                        <execution>
                            <!-- IMPORTANT -->
                            <phase>process-sources</phase>
                            <goals>
                                <goal>compile</goal>
                                <goal>test-compile</goal>
                            </goals>
                        </execution>
                    </executions>
                    <dependencies>
                        <dependency>
                            <groupId>org.aspectj</groupId>
                            <artifactId>aspectjtools</artifactId>
                            <version>${aspectj.version}</version>
                        </dependency>
                    </dependencies>
                </plugin>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>exec-maven-plugin</artifactId>
                    <version>1.4.0</version>
                    <configuration>
                        <mainClass>${main-class}</mainClass>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.dstovall</groupId>
                    <artifactId>onejar-maven-plugin</artifactId>
                    <version>1.4.4</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>one-jar</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <onejarVersion>0.96</onejarVersion>
                        <mainClass>${main-class}</mainClass>
                        <attachToBuild>true</attachToBuild>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>aspectj-maven-plugin</artifactId>
            </plugin>
            <!--
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
            </plugin>
            -->
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <configuration>
                    <mainClass>${main-class}</mainClass>
                    <cleanupDaemonThreads>false</cleanupDaemonThreads>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.dstovall</groupId>
                <artifactId>onejar-maven-plugin</artifactId>
                <configuration>
                    <mainClass>${main-class}</mainClass>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <pluginRepositories>
        <pluginRepository>
            <id>OneJAR googlecode.com</id>
            <url>http://onejar-maven-plugin.googlecode.com/svn/mavenrepo</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.aspectj</groupId>
                <artifactId>aspectjrt</artifactId>
                <version>${aspectj.version}</version>
                <scope>runtime</scope>
            </dependency>
            <dependency>
                <groupId>org.seleniumhq.selenium</groupId>
                <artifactId>selenium-java</artifactId>
                <version>2.48.2</version>
            </dependency>
            <dependency>
                <groupId>io.github.bonigarcia</groupId>
                <artifactId>webdrivermanager</artifactId>
                <version>1.3.0</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjrt</artifactId>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
        </dependency>
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
        </dependency>
    </dependencies>

    <organization>
        <name>Scrum-Master.de - Agile Project Management</name>
        <url>http://scrum-master.de</url>
    </organization>

</project>

Java driver application:

As you can see, the application holds a WebDriver reference needed by the second pointcut only (the first one does not need it, it can find it via target() binding). The Application class also implements Closeable which enables us to use try with resources in the main method, making sure that the driver will be closed outomatically as the Application instance goes out of scope.

package de.scrum_master.app;

import io.github.bonigarcia.wdm.ChromeDriverManager;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

import java.io.Closeable;
import java.io.IOException;

public class Application implements Closeable {
    private final WebDriver driver;

    public Application() {
        ChromeDriverManager.getInstance().setup();
        driver = new ChromeDriver();
    }

    @Override
    public void close() {
        driver.quit();
    }

    public WebDriver getDriver() {
        return driver;
    }

    public void doSomething() {
        driver.get("http://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_input_type_hidden");

        // Button in main frame
        WebElement button = driver.findElement(By.className("seeResult"));
        // Text field in iframe
        driver.findElement(By.name("fname"));
        // Text area in main frame
        driver.findElement(By.id("textareaCode"));
        // Hidden input field in main frame
        driver.findElement(By.name("bt"));
        // Hidden input field in iframe
        WebElement hiddenCountryField = driver.findElement(By.name("country"));

        // Click button in main frame. This *refreshes* the iframe, making all existing
        // references to elements therein (e.g. 'hiddenCountryField') stale
        button.click();

        // Get value of hidden input field after iframe refresh
        System.out.println(driver.findElement(By.name("country")).getAttribute("value"));

        // This alternative would *not* work because the aspect cannot repair a reference
        // to an element which is gone forever because the iframe was refreshed
        // System.out.println(hiddenCountryField.getAttribute("value"));

        // Click submit button in iframe (triggers both advices)
        driver.findElement(By.cssSelector("input[type=submit]")).click();
    }

    public static void main(String[] args) {
        try (Application application = new Application()) {
            application.doSomething();
        }
    }
}

Aspect:

package de.scrum_master.aspect;

import de.scrum_master.app.Application;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

public aspect IFrameSwitcher {
    WebElement around(WebDriver driver, By by) :
        !within(IFrameSwitcher) &&
            call(WebElement WebDriver+.findElement(*)) &&
            target(driver) &&
            args(by)
        {
            System.out.println(thisJoinPoint + " -> " + by);
            WebElement webElement;
            try {
                System.out.print("  Trying main frame -> ");
                driver.switchTo().defaultContent();
                webElement = proceed(driver, by);
                System.out.println("OK");
                return webElement;
            }
            catch (RuntimeException e) {
                System.out.println(e.getClass().getSimpleName());
                for (WebElement iframe : driver.findElements(By.tagName("iframe"))) {
                    try {
                        System.out.print("  Trying iframe " + iframe.getAttribute("id") + " -> ");
                        driver.switchTo().frame(driver.findElement(By.id("iframeResult")));
                        webElement = proceed(driver, by);
                        System.out.println("OK");
                        return webElement;
                    }
                    catch (RuntimeException e2) {
                        System.out.println(e2.getClass().getSimpleName());
                        e = e2;
                    }
                }
                throw e;
            }
        }

    void around(Application application, WebElement webElement) :
        within(Application) &&
        call(void WebElement+.click()) &&
        this(application) &&
        target(webElement)
    {
        System.out.println(thisJoinPoint + " -> " + webElement);
        WebDriver driver = application.getDriver();
        try {
            System.out.print("  Trying main frame -> ");
            driver.switchTo().defaultContent();
            proceed(application, webElement);
            System.out.println("OK");
        }
        catch (RuntimeException e) {
            System.out.println(e.getClass().getSimpleName());
            for (WebElement iframe : driver.findElements(By.tagName("iframe"))) {
                try {
                    System.out.print("  Trying iframe " + iframe.getAttribute("id") + " -> ");
                    driver.switchTo().frame(driver.findElement(By.id("iframeResult")));
                    proceed(application, webElement);
                    System.out.println("OK");
                    return;
                }
                catch (RuntimeException e2) {
                    System.out.println(e2.getClass().getSimpleName());
                    e = e2;
                }
            }
            throw e;
        }
    }
}

Console log:

Here you can see which of the two pointcuts fires when, how after failed tries they are restarted etc.

call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.className: seeResult
  Trying main frame -> OK
call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.name: fname
  Trying main frame -> NoSuchElementException
  Trying iframe google_ads_iframe_/16833175/TryitLeaderboard_0 -> OK
call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.id: textareaCode
  Trying main frame -> OK
call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.name: bt
  Trying main frame -> OK
call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.name: country
  Trying main frame -> NoSuchElementException
  Trying iframe google_ads_iframe_/16833175/TryitLeaderboard_0 -> OK
call(void org.openqa.selenium.WebElement.click()) -> [[ChromeDriver: chrome on XP (5ab9e5f25d169bbc941ab1b08b346c50)] -> class name: seeResult]
  Trying main frame -> OK
call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.name: country
  Trying main frame -> NoSuchElementException
  Trying iframe google_ads_iframe_/16833175/TryitLeaderboard_0 -> OK
Norway
call(WebElement org.openqa.selenium.WebDriver.findElement(By)) -> By.cssSelector: input[type=submit]
  Trying main frame -> NoSuchElementException
  Trying iframe google_ads_iframe_/16833175/TryitLeaderboard_0 -> OK
call(void org.openqa.selenium.WebElement.click()) -> [[ChromeDriver: chrome on XP (5ab9e5f25d169bbc941ab1b08b346c50)] -> css selector: input[type=submit]]
  Trying main frame -> StaleElementReferenceException
  Trying iframe google_ads_iframe_/16833175/TryitLeaderboard_0 -> OK

I hope this helps. Enjoy!