Getting Lots of ANR due to AccessibilityNodeInfo getChild

Question

Getting lot of ANR reports,

"main" prio=5 tid=1 Runnable | group="main" sCount=0 dsCount=0 flags=0 obj=0x72e8a568 self=0xe65da000 | sysTid=20592 nice=0 cgrp=default sched=0/0 handle=0xe9f6b4a8 | state=R schedstat=( 35792446568 9751828904 97371 ) utm=2328 stm=1249 core=2 HZ=100 | stack=0xff6e5000-0xff6e7000 stackSize=8MB | held mutexes= "mutator lock"(shared held) native: pc 00000000002c45b7 /system/lib/libart.so (_ZN3art15DumpNativeStackERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEEiP12BacktraceMapPKcPNS_9ArtMethodEPv+130) native: pc 0000000000355a83 /system/lib/libart.so (_ZNK3art6Thread9DumpStackERNSt3__113basic_ostreamIcNS1_11char_traitsIcEEEEbP12BacktraceMapb+202) native: pc 0000000000351f67 /system/lib/libart.so (_ZNK3art6Thread4DumpERNSt3__113basic_ostreamIcNS1_11char_traitsIcEEEEbP12BacktraceMapb+34) native: pc 00000000003698df /system/lib/libart.so (_ZN3art14DumpCheckpoint3RunEPNS_6ThreadE+654) native: pc 000000000035662b /system/lib/libart.so (_ZN3art6Thread21RunCheckpointFunctionEv+298) native: pc 00000000003bbcff /system/lib/libart.so (_ZN3art16JniMethodFastEndEjPNS_6ThreadE+46) native: pc 0000000000403fdb /system/framework/arm/boot-framework.oat (Java_android_os_Parcel_nativeReadInt__J+114) at android.os.Parcel.nativeReadInt (Native method) at android.os.Parcel.readInt (Parcel.java:1966) at android.os.Parcel.readExceptionCode (Parcel.java:1906) at android.os.Parcel.readException (Parcel.java:1885) at android.accessibilityservice.IAccessibilityServiceConnection$Stub$Proxy.findAccessibilityNodeInfoByAccessibilityId (IAccessibilityServiceConnection.java:447) at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:286) at android.view.accessibility.AccessibilityNodeInfo.getChild (AccessibilityNodeInfo.java:959) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:94) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100) at MyAccessibilityService.collectTextNodes (MyAccessibilityService.java:69) at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:384) at android.accessibilityservice.Accessi bilityService$2.onAccessibilityEvent (AccessibilityService.java:1527) at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage (AccessibilityService.java:1712) at com.android.internal.os.HandlerCaller$MyHandler.handleMessage (HandlerCaller.java:37)

    "main" prio=5 tid=1 Native
      | group="main" sCount=1 dsCount=0 flags=1 obj=0x72e8a568 self=0xe65da000
      | sysTid=11518 nice=0 cgrp=default sched=0/0 handle=0xe9f6b4a8
      | state=S schedstat=( 34572861602 18459176949 199877 ) utm=2350 stm=1106 core=1 HZ=100
      | stack=0xff6e5000-0xff6e7000 stackSize=8MB
      | held mutexes=
      native: pc 0000000000018dac  /system/lib/libc.so (syscall+28)
      native: pc 00000000000b3729  /system/lib/libart.so (_ZN3art17ConditionVariable16WaitHoldingLocksEPNS_6ThreadE+88)
      native: pc 00000000003bbbe3  /system/lib/libart.so (_ZN3artL12GoToRunnableEPNS_6ThreadE+306)
      native: pc 00000000003bba81  /system/lib/libart.so (_ZN3art12JniMethodEndEjPNS_6ThreadE+8)
      native: pc 00000000007ca401  /system/framework/arm/boot-framework.oat (Java_android_os_BinderProxy_transactNative__ILandroid_os_Parcel_2Landroid_os_Parcel_2I+144)
      at android.os.BinderProxy.transactNative (Native method)
      at android.os.BinderProxy.transact (Binder.java:748)
      at android.accessibilityservice.IAccessibilityServiceConnection$Stub$Proxy.findAccessibilityNodeInfoByAccessibilityId (IAccessibilityServiceConnection.java:446)
      at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:286)
      at android.view.accessibility.AccessibilityNodeInfo.getChild (AccessibilityNodeInfo.java:959)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:94)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100)
      at MyAccessibilityService.collectTextNodes (MyAccessibilityService.java:69)
      at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:384)
      at android.accessibilityservice.AccessibilityService$2.onAccessibilityEvent (AccessibilityService.java:1527)
      at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage (AccessibilityService.java:1712)
      at com.android.internal.os.HandlerCaller$MyHandler.handleMessage (HandlerCaller.java:37)
      at android.os.Handler.dispatchMessage (Handler.java:105)


    "main" prio=5 tid=1 Runnable
      | group="main" sCount=0 dsCount=0 flags=0 obj=0x73bf7568 self=0xe7445000
      | sysTid=18941 nice=0 cgrp=default sched=0/0 handle=0xeae2e4a8
      | state=R schedstat=( 169234820933 190791369297 635928 ) utm=11752 stm=5170 core=0 HZ=100
      | stack=0xff795000-0xff797000 stackSize=8MB
      | held mutexes= "mutator lock"(shared held)
      at java.util.ArrayList.clear (ArrayList.java:565)
      at android.view.accessibility.AccessibilityNodeInfo.init (AccessibilityNodeInfo.java:3211)
      at android.view.accessibility.AccessibilityNodeInfo.obtain (AccessibilityNodeInfo.java:3015)
      at android.view.accessibility.AccessibilityCache.getNode (AccessibilityCache.java:231)
    - locked <0x0f41ae94> (a java.lang.Object)
      at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:272)
      at android.view.accessibility.AccessibilityNodeInfo.getChild (AccessibilityNodeInfo.java:959)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:84)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:85)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:85)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:85)
      at MyAccessibilityService.collectTextNodes (MyAccessibilityService.java:66)
      at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:360)
      at android.accessibilityservice.AccessibilityService$2.onAccessibilityEvent (AccessibilityService.java:1527)
      at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage (AccessibilityService.java:1712)
      at com.android.internal.os.HandlerCaller$MyHandler.handleMessage (HandlerCaller.java:37)
      at android.os.Handler.dispatchMessage (Handler.java:105)

"main" prio=5 tid=1 TimedWaiting
  | group="main" sCount=1 dsCount=0 flags=1 obj=0x73228568 self=0xed55a000
  | sysTid=19996 nice=0 cgrp=default sched=0/0 handle=0xed8414a8
  | state=S schedstat=( 39022541552 53623486893 130046 ) utm=3252 stm=648 core=2 HZ=100
  | stack=0xff636000-0xff638000 stackSize=8MB
  | held mutexes=
  at java.lang.Object.wait (Native method)
- waiting on <0x0b44b730> (a java.lang.Object)
  at java.lang.Object.wait (Object.java:422)
  at android.view.accessibility.AccessibilityInteractionClient.waitForResultTimedLocked (AccessibilityInteractionClient.java:687)
  at android.view.accessibility.AccessibilityInteractionClient.getFindAccessibilityNodeInfosResultAndClear (AccessibilityInteractionClient.java:582)
- locked <0x0b44b730> (a java.lang.Object)
  at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:291)
  at android.view.accessibility.AccessibilityInteractionClient.getRootInActiveWindow (AccessibilityInteractionClient.java:160)
  at android.accessibilityservice.AccessibilityService.getRootInActiveWindow (AccessibilityService.java:572)
  at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:370)

My code

ArrayList<AccessibilityNodeInfo> collectNodes(AccessibilityNodeInfo node) {

    ArrayList<AccessibilityNodeInfo> nodeInfoArrayList = new ArrayList<>();
    try {
        int childCount = node.getChildCount();
        for (int index = 0; index < childCount; index++) {
            AccessibilityNodeInfo childNode = node.getChild(index);


            traverseNode(childNode);
            if (childNodes != null && childNodes.size() > 0) {
                nodeInfoArrayList.addAll(childNodes);
                childNodes.clear();
            }
        }
    }
    catch (Exception e){
        e.printStackTrace();
    }
    return nodeInfoArrayList;
}

private void traverseNode(AccessibilityNodeInfo node) {
    try {
        AccessibilityNodeInfo edittextNode = null;
        if (null == node)
            return;

        final int count = node.getChildCount();
        if (count > 0) {
            for (int i = 0; i < count; i++) {
                AccessibilityNodeInfo childNode = node.getChild(i);
                if (childNode == null) {
                    node.recycle();
                    return;
                }
                else {
                    traverseNode(childNode);
                }
            }
        } else {

        }
    }
    catch (Exception error){
        error.printStackTrace();
    }
}

even more issues in Android 8.0. Any idea how to deal with these ANR's.

config.xml file

<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"

    android:accessibilityEventTypes="typeWindowStateChanged|typeViewFocused|typeWindowContentChanged"

    android:notificationTimeout="100"
    android:canRetrieveWindowContent="true"

    android:accessibilityFeedbackType="feedbackGeneric"
    android:accessibilityFlags="flagReportViewIds|flagRequestEnhancedWebAccessibility"
    android:description="Test"/>

please note almost all the ANR report from Android 8.0 devices.

Answer 1

1. The first important thing is to recycle all the nodes allocated by getChild , findAccessibilityNodeInfoByAccessibilityId or any other function. So you should have something like this:

 AccessibilityNodeInfo findFirstViewByText(AccessibilityNodeInfo rootNode, String text) { if (rootNode == null) return null; if (rootNode.getText() != null && rootNode.getText().toString().equals(text)) return rootNode; int childCount = rootNode.getChildCount(); for (int i = 0; i < childCount; i++) { AccessibilityNodeInfo tmpNode = rootNode.getChild(i); if (tmpNode != null) { AccessibilityNodeInfo res = findFirstViewByText(tmpNode, text); if (res != null) { return res; } else { tmpNode.recycle(); } } } return null; } } node = findFirstViewByText(rootNode, "something"); if(node != null) { node.performAction(AccessibilityNodeInfo.ACTION_CLICK); node.recycle(); } 

2. Keep in mind that onAccessibilityEvent may be called too often while you interact with the target application and also your processing in this function is blocking UI Thread on the target application. You can overcome this by:

   • Throttling the calls to onAccessibilityEvent ie don't execute your code if the last execution happened less than x milliseconds ago, but make sure that last call will trigger execution of your code.
   • Improve the performance of your code by avoiding unnecessary traversing. Define states so the app will look only for the nodes relevant for the current state.

3. Feel free to share more of your code so i/we can understand what exactly is your goal, that way we can be more helpful. As i can see from your log, there are several threads traversing the virtual dom of accessibility service and they are causing deadlock.

Answer 2

I'm highly suspicious of your collect nodes logic. It's very convolluted. It actually took me a good while to confirm that it was doing what I thought it was doing, and I'm still not convinced, though mostly because there are a few class/global variables that you have not shared, keep ing me from being 100% sure of anything. Allow me to describe what I think it's doing:

CollectNodes: Collects all descendants of the given node, and returns them in a list.

So, given that description, let me show you a simpler implementation.

public ArrayList<AccessibilityNodeInfo> collectNodes() {
    ArrayList<AccessibilityNodeInfo> results = new ArrayList<>();

    try {
        collectNodes(getRootInActiveWindow(), results);    
    } catch (Exception e) {
        //Handle exceptions, though, if you don't do stupid things exceptions shouldn't happen
    }

    return results;
}

private void collectNodes(AccessibilityNodeInfo nodeInfo, ArrayList<AccessibilityNodeInfo> nodes) {

    if (nodeInfo == null) return;

    if (someFilteringFunctionYouWantToApplyToLimitNodes(nodeInfo)) {
        nodes.add(nodeInfo);
    }

    for (int i = 0; i < nodeInfo.getChildCount(); i++) {
        collectNodes(nodeInfo.getChildAt(i), nodes);
    }
}

NOW, let's talk about some of the things, actually really only one specific thing that I removed from your implementation, and that is this line:

node.recycle();

I believe this is the actual line that is causing you problems. At no point in your code have you created any of your own AccessibilityNodeInfo objects. All of the objects that you're using belong to the the hierarchy of nodes that has been created for you by the Operating system. These nodes should be viewed as a READ ONLY state of your application at a point in time. You are recycling things and reusing them.