NSOpenPanel打破了macOS上的UI测试

Question

I'm using Xcode to do UI testing on a sandboxed macOS app that has the com.apple.security.files.user-selected.read-write entitlement (ie, can access files and folders explicitly selected by the user via an NSOpenPanel GUI).

I have noticed that code coverage stops right after the open panel is presented modally. This is my code:

@IBAction func go(_ sender: Any) {

    let panel = NSOpenPanel()
    panel.canCreateDirectories = true
    panel.canChooseDirectories = true
    panel.canChooseFiles = false
    panel.allowsMultipleSelection = false

    let response = panel.runModal()

    switch response {
    case NSApplication.ModalResponse.OK:
        openPanelDidSelectURL(panel.urls[0])

    default:
        return
    }
}

(I have recorded my UI tests so that the NSOpenPanel is accepted right away, choosing the folder where it was open.)

Code coverage ends up highlighted like this:

I have tried replacing the switch statement with a fatalError() call, but the UI test still completes successfully, suggesting that anything immediately after:

let response = panel.runModal()

...is not executed during the test.

Disabling sandboxing seems to have no effect, so I suspect it is running the open panel modally that causes trouble...

Answer 1

I tried all other available methods for presenting the open panel, namely:

panel.begin { (response) in
    switch response {
    case NSApplication.ModalResponse.OK:
        self.openPanelDidSelectURL(panel.urls[0])

    default:
        return
    }
}

...and also:

panel.beginSheetModal(for: view.window!) { (response) in
    switch response {
    case NSApplication.ModalResponse.OK:
        self.openPanelDidSelectURL(panel.urls[0])

    default:
        return
    }
}

...but the result is always the same: All code immediately after presenting the panel is not covered during tests.

---

In the end , I realized that my UI tests cannot rely on some user-selectable folder being present wherever the open panel lands (last visited directory?), so I opted for using mocking instead.

First, in my UI test classes, I adopted this setup logic:

override func setUp() {
    continueAfterFailure = false
    let app = XCUIApplication()
    app.launchArguments.append("-Testing")
    app.launch()
}

(the hyphen before "Testing" is mandatory, otherwise my document-based macOS app will think I am launching it to open a document named "Testing", and fail to do so)

Next, On the app side, I defined a global computed property to determine whether we are running under a test or not:

public var isTesting: Bool {
    return ProcessInfo().arguments.contains("-Testing")
}

Finally, also on the app side I wrapped all NSOpenPanel calls into two methods: One for prompting the user for input files to read, and another to prompt the user for an output directory into which to write the resulting files (this is all my app needs from NSOpenPanel ):

public func promptImportInput(completionHandler: @escaping (([URL]) -> Void)) {
    guard isTesting == false else {
        /* 
          Always returns the URLs of the bundled resource files: 
           - 01@2x.png, 
           - 02@2x.png, 
           - 03@2x.png,
             ...
           - 09@2x.png, 
         */
        let urls = (1 ... 9).compactMap { (index) -> URL? in
            let fileName = String(format: "%02d", index) + "@2x"
            return Bundle.main.url(forResource: fileName, withExtension: "png")
        }
        return completionHandler(urls)   
    }
    // (The code below cannot be covered during automated testing)

    let panel = NSOpenPanel()
    panel.canChooseFiles = true
    panel.canChooseDirectories = true
    panel.canCreateDirectories = false
    panel.allowsMultipleSelection = true

    let response = panel.runModal()

    switch response {
    case NSApplication.ModalResponse.OK:
        completionHandler(panel.urls)
    default:
        completionHandler([])
    }
}

public func promptExportDestination(completionHandler: @escaping((URL?) -> Void)) {
    guard isTesting == false else {
        // Testing: write output to the temp directory 
        // (works even on sandboxed apps):
        let tempPath = NSTemporaryDirectory()
        return completionHandler(URL(fileURLWithPath: tempPath))
    }
    // (The code below cannot be covered during automated testing)

    let panel = NSOpenPanel()
    panel.canChooseFiles = false
    panel.canChooseDirectories = true
    panel.canCreateDirectories = true
    panel.allowsMultipleSelection = false

    let response = panel.runModal()

    switch response {
    case NSApplication.ModalResponse.OK:
        completionHandler(panel.urls.first)
    default:
        completionHandler(nil)
    }
}

The portions of these two functions that use the actual NSOpenPanel instead of mocking the user-selected files/directories are still excluded from gathering code coverage statistics (but this time, it's by design).

But at least now it's just this two places. The rest of my code just calls these two functions and does no longer interact with NSOpenPanel directly. I have 'abstracted' the OS's file browsing interface away from my app...