Ways to avoid having to sleep()/delay() your Katalon Studio code

Question

I have these test cases, see WebUI.delay() and sleep() everywhere, but they break as soon as I remove those delays...

How can I avoid having to delay my test cases?

Answer 1

There are very few legitimate cases where you would want to delay() your test case, or put it to sleep() ...

Main legit use case for this, is to poll an API until you get some expected result from it...or to do some performance testing (namely to simulate an impatient user).

It is a code smell, because it:

• is literally stopping the execution of your test case for some hard-coded amount of time, which means:
• a lot of wasted time, or
• situations where "it works on my machine (but not yours)!"
• is a brute force solution to something where either a built-in waitForXXX() method exist, or some simple custom keyword can be written to solve the problem

Yet there are so many people using those methods because there seems to be no other choice...

How can we prevent these?

Let's go look at some examples:

Examples of preventable uses of delay() / sleep()

Keep in mind, most if not all of these are based on what I've had to face in testing a Zoho app, and based on my complete overhauling of the Katalon project for that...

With that in mind, let's take a look at the first use case I encountered when being tasked with that project...

Elements that are supposed to be on the page (especially after page load)

Bad way

The laziest way is to just do this:

WebUI.navigateTo(homePage);

WebUI.delay(5);

WebUI.click(findTestObject("Home page/Dashboard item"));

Better way

This is better way to do it...

WebUI.navigateTo(homePage);

final TestObject loader = findTestObject("Home page/loader");

WebUI.waitForElementVisible(loader, 2, FailureHandling.OPTIONAL);

WebUI.waitForElementNotVisible(loader, 5); 

final TestObject dashboardItem = findTestObject("Home page/Dashboard item");

WebUI.waitForElementVisible(dashboardItem, 2);

WebUI.click(dashboardItem);

Custom widgets

Auto-complete

In Zoho (namely CRM), an auto-complete has three major parts:

• text field
• loading icon
• dropdown list

Bad way

A noob may handle an autocomplete like:

WebUI.sendKeys(findTestObject("Page/My Autocomplete/input field"), "some text");

WebUI.delay(3); // DON'T DO THIS, PLEASE!!

WebUI.click(findTestObject("Page/My Autocomplete/first dropdown option"));

Heaven forbid it be a more complicated use case, where the autocomplete be fetch-on-scroll, and the dropdown option you're looking for be on the nth page, for n > 1.

Then what will you do? Some scroll, then some hard-coded delay() , inside for loop?

Better way

This strategy-based way involves a lot more code, but is way more correct:

We're going to create Custom Keyword for it!

public final class GeneralWebUIUtils { 
    public static void HandleAutoComplete(TestObject textField, String input, TestObject loader, TestObject dropdownOption, BaseSelectionStrategy strategy = null) throws StepFailedException {
        WebUI.click(textField)

        WebUI.sendKeys(textField, input)

        TimeLoggerUtil.LogAction({
            return WebUI.waitForElementNotVisible(loader, 3)
        },
        "Loader",
        "disappear");

        TimeLoggerUtil.LogAction({
            return WebUI.waitForElementPresent(dropdownOption, 3, FailureHandling.STOP_ON_FAILURE);
        },
        "Dropdown option",
        "become present")

        BaseSelectionStrategy selectionStrategy = strategy;
        if (strategy == null)
            selectionStrategy = new BaseSelectionStrategy(input);

        selectionStrategy.doSelect(dropdownOption);
    }

    public static boolean WaitForElementCondition(Closure<Boolean> onCheckCondition, Closure onContinue, TestObject to, int timeOut, FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
        final long startTime = System.currentTimeMillis()
        boolean isConditionSatisfied = false;
        while ((System.currentTimeMillis() < startTime + timeOut * 1000) && (!isConditionSatisfied)) {
            isConditionSatisfied = WebUI.waitForElementPresent(to, 1, failureHandling) && onCheckCondition(to);
            if (onContinue != null)
                onContinue(isConditionSatisfied, to);
        }
        if ((!isConditionSatisfied) && (failureHandling.equals(FailureHandling.STOP_ON_FAILURE))) {
            KeywordUtil.markFailedAndStop("Condition for TestObject '${to.getObjectId()}' not met after ${(System.currentTimeMillis() - startTime) / 1000} seconds");
        }
        return isConditionSatisfied;
    }

    public static boolean WaitForElementHasText(TestObject to, String expectedText, int timeOut, FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
        return this.WaitForElementCondition({ TestObject testObj ->
            return WebUI.getText(testObj).contains(expectedText);
        }, { boolean success, TestObject testObj ->
            if (!success) {
                WebUI.waitForElementNotPresent(testObj, 1, FailureHandling.OPTIONAL);

                WebUI.waitForElementPresent(testObj, timeOut);
            }
        },
        to,
        timeOut,
        failureHandling);
    }
}

public class BaseSelectionStrategy {
    protected final String input;

    public BaseSelectionStrategy(String input) {
        this.input = input;
    }

    public void doSelect(TestObject dropdownOption) {
        this.waitForDropdownOption(dropdownOption);
        WebUI.click(dropdownOption);
    }

    public boolean waitForDropdownOption(TestObject dropdownOption, FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
        return GeneralWebUIUtils.WaitForElementHasText(dropdownOption, input, this.getWaitTime(), failureHandling);
    }

    public int getWaitTime() {
        return 5;
    }
}

// this is only here to allow time-logging of an action
public final class TimeLoggerUtil { 
    public static boolean LogAction(Closure<Boolean> onAction, String elementDesc, String expectationDesc) throws StepFailedException {
        final long startTime = System.currentTimeMillis();

        try {
            if (onAction()) {
                KeywordUtil.logInfo("${elementDesc} took ${(System.currentTimeMillis() - startTime) / 1000} seconds to ${expectationDesc}");
                return true;
            } else {
                KeywordUtil.markWarning("${elementDesc} didn't ${expectationDesc} after ${(System.currentTimeMillis() - startTime) / 1000} seconds");
            }
        } catch (StepFailedException ex) {
            KeywordUtil.markFailedAndStop("${elementDesc} didn't ${expectationDesc} after ${(System.currentTimeMillis() - startTime) / 1000} seconds...\n${ex.getMessage()}");
            throw ex;
        }

        return false;
    }
}

Then, to use it, simply be like:

GeneralWebUIUtils.HandleAutoComplete(
    findTestObject("Page/My Autocomplete/input field"),
    "some text", // or whatever
    findTestObject("Page/My Autocomplete/loader"),
    findTestObject("Page/My Autocomplete/first dropdown option"),
);

I will admit, this is a lot more code, but is way more of a robust solution as it will not hang your test case up!

What, your autocomplete is fetch-on-scroll? No problem!

Simply create a BaseScrollableSelectionStrategy (and an ActionHandler ):

@InheritConstructors
// feel free to extend this class as you see fit
public class BaseScrollableSelectionStrategy extends BaseSelectionStrategy {

    @Override
    public void doSelect(TestObject dropdownOption) {
        final long startTime = System.currentTimeMillis();

        ActionHandler.Handle({
            this.waitForDropdownOption(dropdownOption);
        }, { boolean success, _ ->
            KeywordUtil.logInfo("${dropdownOption.getObjectId()} ${this.getActionStatus(success, startTime)}")
            if (!success) {
                final TestObject lastAvailableDropdownItem = new TestObject("Last available dropdown item")
                        .addProperty("xpath",
                        ConditionType.EQUALS,
                        "//lyte-drop-box[not(contains(concat(' ', @class, ' '), ' lyteDropdownHidden '))]//lyte-drop-item[last()]"); // TODO: change this to the XPath for the last dropdown option on your auto complete drop-down

                TimeLoggerUtil.LogAction({
                    if (!WebUI.waitForElementPresent(lastAvailableDropdownItem, 2))
                        return false;

                    GeneralWebUIUtils.ScrollDropdownOptionIntoView(lastAvailableDropdownItem);

                    return true;
                }, lastAvailableDropdownItem.getObjectId(),
                "scroll into view");
            }
        }, 15)
        GeneralWebUIUtils.ScrollDropdownOptionIntoView(dropdownOption);
        WebUI.click(dropdownOption);
    }

    protected String getActionStatus(boolean success, long startTime) {
        if (success)
            return "took ${(System.currentTimeMillis() - startTime) / 1000} seconds to show up";
        return "didn't show up after ${(System.currentTimeMillis() - startTime) / 1000} seconds";
    }

    @Override
    public int getWaitTime() {
        return 2;
    }
}

public class ActionHandler {
    public static void Handle(Closure onAction, Closure onDone, long timeOut) {
        long startTime = System.currentTimeSeconds();
        while (System.currentTimeSeconds() < startTime + timeOut) {
            try {
                onDone(true, onAction());
                return;
            } catch (Exception ex) {
                onDone(false, ex);
            }
        }
    }
}

add a keyword for scrolling the dropdown option into view:

public static void ScrollDropdownOptionIntoView(TestObject to) {
    WebUI.executeJavaScript("arguments[0].scrollIntoView({block: 'center'})", [WebUiCommonHelper.findWebElement(to, 3)])

    WebUI.waitForElementVisible(to, 2)
}

and then your use case should look like:

final String textFieldInput = "some text" // or whatever

GeneralWebUIUtils.HandleAutoComplete(
    findTestObject("Page/My Autocomplete/input field"),
    textFieldInput,
    findTestObject("Page/My Autocomplete/loader"),
    findTestObject("Page/My Autocomplete/first dropdown option"),
    new BaseScrollableSelectionStrategy(textFieldInput), // or whatever your derived scrollable selection strategy is
);

New Table Row

This is yet another unjustified use case that I had to deal with, this one much earlier on in the project. It was something like this:

Bad way

WebUI.click(findTestObject("Page/Add Button"));

WebUI.delay(3);

WebUI.click(findTestObject("Page/Second Row Dropdown button"));

Better way

WebUI.click(findTestObject("Page/Add Button"));

final TestObject secondRowDropdownBtn = findTestObject("Page/Second Row Dropdown button");

WebUI.waitForElementPresent(secondRowDropdownBtn, 3);

WebUI.scrollToElement(secondRowDropdownBtn, 2);

WebUI.click(secondRowDropdownBtn);

...or better yet...

WebUI.click(findTestObject("Page/Add Button"));

final TestObject secondRowDropdown = findTestObject("Page/Second Row Dropdown button");

WebUI.waitForElementPresent(secondRowDropdown, 3);

WebUI.scrollToElement(secondRowDropdown, 2);

WebUI.click(findTestObject("Page/Second Row Dropdown button"));

Saving a model to be created, and then doing/checking something on the next page

This is a common one. You are about to click "Submit", to create some record (item/rate card/discount/member/...)...

You wait for the button to disable...

...then you try that WebUI.waitForPageLoad(15); between waiting for the button to disappear, and waiting for the next page to fully load...

Test case fails. WHY.png

Bad way

You pull your hair out in frustration, and then give into that temptation to instead do WebUI.delay(15) ...

It works, but it's slow?!

You now have to run that test case to create 10+ items/rate cards/discounts/members/... per a data file...

AnyMinuteNow.gif

Better way

This way handles both the save button and the URL change. It will require more custom keywords to our GeneralWebUIUtils :

public final class GeneralWebUIUtils { 
    private static boolean WaitForURLCondition(Closure<Boolean> onCheckCondition, int timeOut, Closure<String> onErrorMessage, FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
        final long startTime = System.currentTimeMillis()
        boolean isConditionSatisfied = false;
        while ((System.currentTimeMillis() < startTime + timeOut * 1000) && (!isConditionSatisfied)) {
            isConditionSatisfied = onCheckCondition(WebUI.getUrl())
        }
        if ((!isConditionSatisfied) && (failureHandling.equals(FailureHandling.STOP_ON_FAILURE))) {
            KeywordUtil.markFailedAndStop("${onErrorMessage(WebUI.getUrl())} after ${(System.currentTimeMillis() - startTime) / 1000} seconds");
        }
        return isConditionSatisfied;
    }

    public static boolean WaitForURLNotEquals(String url, int timeOut, FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
        return this.WaitForURLCondition({ String browserURL ->
            return !(browserURL =~ SMDStringUtils.GetURLPattern(url)).matches();
        },
        timeOut, { String browserURL ->
            "URL '${browserURL}' matches unexpected '${url}'"
        },
        failureHandling)
    }

    public static boolean WaitForURLEquals(String url, int timeOut, FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
        return this.WaitForURLCondition({ String browserURL ->
            return (browserURL =~ SMDStringUtils.GetURLPattern(url)).matches();
        },
        timeOut, { String browserURL ->
            "URL '${browserURL}' does not match expected '${url}'"
        },
        failureHandling)
    }

public static void HandleSaveButton(TestObject saveButton) throws StepFailedException {
        this.HandleSaveButton(saveButton, true);
    }

    public static void HandleSaveButton(TestObject saveButton, boolean shouldSuccessfullySave) throws StepFailedException {
        WebUI.scrollToElement(saveButton, 3)

        TimeLoggerUtil.LogAction({
            return WebUI.waitForElementClickable(saveButton, 3, FailureHandling.STOP_ON_FAILURE);
        },
        "Save button",
        "become clickable");

        WebUI.click(saveButton)

        TimeLoggerUtil.LogAction({
            return WebUI.waitForElementHasAttribute(saveButton, GeneralWebUIUtils.DISABLED, 5, FailureHandling.STOP_ON_FAILURE);
        },
        "Save button",
        "disable");

        if (shouldSuccessfullySave) {
            TimeLoggerUtil.LogAction({
                return WebUI.waitForElementNotPresent(saveButton, 5, FailureHandling.STOP_ON_FAILURE);
            },
            "Save button",
            "disappear from the DOM");

            TimeLoggerUtil.LogAction({
                WebUI.waitForPageLoad(5);
                return true;
            },
            "Page",
            "load")
        }
    }
}

public final class SMDStringUtils { 
    public static String GetURLPattern(String url) {
        return (/^(http(s)?\:\/\/)?${url}(\/)?$/);
    }
}

In the test case, we just do:

// submit the changes
GeneralWebUIUtils.HandleSaveButton(findTestObject('Page/Submit button'))

// verify that the save happened successfully
GeneralWebUIUtils.WaitForURLNotEquals(creationPageURL, 15) // creationPageURL is the URL of the creation page you were just on

or better yet, if you had to click some "Add" button on some initial page, and submit button is supposed to end up sending you back to that initial page:

// submit the changes
GeneralWebUIUtils.HandleSaveButton(findTestObject('Page/Submit button'))

// verify that the save happened successfully
GeneralWebUIUtils.WaitForURLEquals(initPageURL, 15) // initPageURL is the URL of the initial page you were just on

Thoughts? Concerns?

Let me know in the comments section below!