What are the best practices for structuring an XCTest UI Test Suite?

Question

I am setting up a test suite for an iOS app. I am using Xcode's XCTest framework. This is just a UI test suite. Right now I have a test target with one file TestAppUITests. All of my test cases reside in this file (first question: should all of my test cases be in here or do I need more files?) Within this file, I have a bunch of test cases that work as if it's a user using the app. Logging in/ Create account then login -> Navigate around app -> Check if UI elements loaded -> Add additional security like secondary email address -> logout. How should these be ordered?

I've researched all around, found some gems here and there but still have questions. Within the test target, should you have multiple files? I just have the one within my UITest target. My other big question is a bit harder to explain. Each time we run the test suite from the beginning the app starts in a state where you are not logged in, so, for example, in order to test something like navigate WITHIN the app, I need to run that test to login first. Right now I have it setup so that the login test runs once, then all other tests after it, then ends with logout. But that one file TestAppUITests is getting very long with tons of test cases. Is this best practice?

Answer 1

Soo. Lets divide this into more parts:

1/ Should all of my test cases be in here or do I need more files?

Well - your tests are same as any other app code, you have. Do you have all of the app code in one file? Probably not, so a good practice is to divide your tests into more classes (I do it in the way of what they test - LoginTests class, UserProfileTests class etc etc.).

To go further with this - I have test classes and methods divided into separate files - fe I have a method, that will do a login in UI test, so I have the method in UITestCase+Login extension ( UITestCase is a class, that is extended by all these UITestCase+Something extensions) and I have the test, that will do the login in LoginTests in which I am calling the login method from the UITestCase+Login extension.

However - you don't necessarily need more test classes - if you decide to have all of your UI tests in one class, thats your choice. Everything's gonna work, but having the tests and methods that these tests use in separate files is just a good practice for future development of tests and methods.

2/ ... Add additional security like secondary email address... How should these be ordered?

Order them into methods and call these in tests.

This is my method for expecting some UI message, when I use invalid login credentials:

func expectInvalidCredentialsErrorMessageAfterLoggingWithInvalidBIDCredentials() {
    let alertText = app.alerts.staticTexts[localizedString("mobile.account.invalid_auth_credentials")]
    let okButton = app.alerts.buttons[localizedString("common.ok")]

    wait(
        until: alertText.exists && okButton.existsAndHittable,
        timeout: 15,
        or: .fail(message: "Alert text/button are not visible.")
    )
    okButton.tap()
}

And this is my use of it in test:

func testTryToLoginWitMissingBIDAndExpectError() {
    let inputs = BIDLoginInputs(
        BID: "",
        email: "someemail@someemail.com",
        departureIATA: "xxx",
        dayOfDeparture: "xx",
        monthOfDeparture: "xxx",
        yearOfDeparture: "xxx"
    )

    loginWithBIDFromProfile(inputs: inputs)
    expectInvalidCredentialsErrorMessageAfterLoggingWithInvalidBIDCredentials()
}

You can see, that tests are very readable and that they (almost fully) consists of methods, which are re-usable in more tests.

3/ Within the test target, should you have multiple files?

Again - this is up to you, however having everything in one file is not really good for maintenance and future development of these tests.

4/ ... Each time we run the test suite from the beginning the app starts in a state where you are not logged in...Right now I have it setup so that the login test runs once, then all other tests after it, then ends with logout...

Not a good approach (in my humble point of view ofc.) - put the functionality into methods (yes, I repeat myself here :-) ) and divide the test cases into more files (ideally by the nature of their functionality, by "what they do").

Hope, this helped you, I struggled a lot with the same questions, when I was starting with iOS UI tests too. Oh. And btw - my article on medium.com about advanced tactics and approaches to iOS UI tests with XCTest is coming out in a few days, I can put a link to it, once it is out - that should help you even further.

Answer 2

Echoing this answer , it is against best practices to store all of your tests for a complex application in a single file, and your tests should be structured so that they are independent of each other, only testing a single behaviour in each test.

It may seem counter-intuitive to split everything up into many tests which need to go through re-launching your app at the beginning of every test, but this makes your test suite more reliable and easier to debug as the number of unknowns in a test is minimised by having smaller, shorter tests. Since UI tests take a relatively long time to run, which can be frustrating, you should try to minimise the amount that you need by ensuring your app has good unit/integration test coverage.

With regards to best practices on structuring your XCTest UI tests, you should look into the Screenplay or Page Object models. The Page Object model has been around a long time, and there are many posts around it although many tend to be focused on Selenium or Java-based test frameworks. I have written posts on both the Page Object model and the Screenplay model using Swift and XCTest, they should help you.