Java Swing based windows desktop application is not detected by AccessibilityInsights

Question

We have developed a Windows based Desktop application using Java Swing and build a exe of it using javapackager available in Java 8.

Now we are trying to automate the testing of it using Karate-Robot but Robot could not detect the fields on the GUI of Application. To further dig we have installed AccessibilityInsights suggested by Karate Community to figure out how to access elements on the GUI but it seems that our application is not getting detcetd in it. See attached screenshot.

Can someone help us on how to fix the application?

Will Java Accessibility Bridge help here? As per documentation https://docs.oracle.com/en/java/javase/11/access/java-accessibility-guide.pdf , Application needs to implement Java Accessibility APIs when it needs to be available to Windows Assistive technologies.

Edit 1: As per https://docs.oracle.com/javase/tutorial/uiswing/misc/access.html , "Because support for the Accessibility API is built into the Swing components, your Swing program will probably work just fine with assistive technologies, even if you do nothing special."

Answer 1

Ashish--

There have been multiple iterations of Windows-based assistive technologies over the years. The current iteration has existed since 2005 and is called UI Automation . This is the technology that is required for OS-provided tools like Magnifier and Narrator. It's also the technology that is required by Accessibility Insights for Windows. From your description, it might also be the technology that is required by Karate-Robot.

Java Accessibility Bridge uses an older iteration of assistive technology that is no longer recognized by OS-provided accessibility tools. It is recognized by some third-party accessibility tools (JAWS and NVDA, for example), but it is not recognized by OS-provided accessibility tools. To the best of my knowledge, Oracle has no plans to support UI Automation in the future. When their docs say that your apps will "work just fine", they're referring to users who rely on these third-party tools.

Accessibility Insights for Windows has made a conscious decision to support only UI Automation, since supporting 2 platforms would make things much more difficult. We chose to go with UI Automation because it is the only technology that is:

• Actively supported by the operating system, with new features being regularly added and documented.
• Supported by native Windows controls and most actively supported language frameworks.
• The technology that we would expect new assistive technology tools to implement first.

I'm sorry to be the bearer of bad news on this.

DaveTryon (Accessibility Insights team)