Android UI Automation Without Computer

Question

Hi i want to automate a simple task in my android phone, which involves checking all check boxes on a single app page (app does not have check all feature).

Is there any way to automate UI in android like you can do for browsers using selenium?

I have some idea of Android test automation tools but i do not want to connect my phone to computer each time. I am looking for a way to do it from my mobile. Maybe some library that allows to do this upon which i can build a simple android app?

If there is no such library or tool can you recommend some alternative?

EDIT: (note: i am not an Android Developer so i apologize in advance if i explain something wrong) Found a way the solution is in Answers.

Answer 1

(note: i am not an Android Developer so i apologize in advance if i explain something wrong)

So using Accessibility service android automation can be done. Following is some detail on how to use this service for automation.

First you implement the accessibility service class for your app

public class SomeAppAccessibilityService extends AccessibilityService {


@Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent) {
    //if some event occurs this method is called you can implement appropriate action here when you send a custom event from your app code

}


@Override
public void onInterrupt() {

 }
}

After that declare it in manifest like any other service. Example of service declaration

<service android:name=".service.SomeAppAccessibilityService"
             android:label="some app Accessibility Service"
             android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" >
        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/someappservice"/>
        <intent-filter>
            <action android:name="android.accessibilityservice.AccessibilityService"/>
        </intent-filter>
    </service>

You can google what different parameters mean and more. Some basic explanation is as follows: android:name (here you tell service where your SomeAppAccessibilityService class is) <meta-data android:resource= (where your xml file containing accessibility service properties is)

Example of resources xml file for properties:

<?xml version="1.0" encoding="utf-8"?><accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes = "typeAllMask"
    android:packageNames="com.example.app1,com.example.app2"
    android:accessibilityFeedbackType="feedbackGeneric"
    android:notificationTimeout="100"
    android:canRetrieveWindowContent="true"
    />

After all that is done now you can look for accessibility events in onAccessibilityEvent method as follows

if(accessibilityEvent.getEventType() == AccessibilityEvent.TYPE_ANNOUNCEMENT) {} //for example i want to capture announcement event

When some accessibility event (could be number of things like click,long click, announcement or even custom triggered event) happens,OnAccessibilityEvent method is called and we can use that to get current screen info through accessibility service using

AccessibilityNodeInfo info=getRootInActiveWindow(); //root view node info
AccessibilityNodeInfo childElem = info.getChild(n); //for getting nth child of root node
info.getChildCount(); //to get total size of child nodes
//then you can iterate on those children to get their child nodes
childElem.performAction(AccessibilityNodeInfo.ACTION_CLICK); //to perform some action on child elem like click as shown here
//there are other many useful AccessibilityNodeInfo class method that you can use

To send a custom event from your app or some other service. for example i have drawn overlay on some third party app (lets say called App1) i want to automate, using my own app. When i click some button on overlay i would want that some action is executed on that third party app's (App1) current window. To execute some action accessibility service need to capture some event. Either you can capture raw events like click etc or you can send custom events (there may be a more refined method to do this out there but at the time of post i did not find any). Here is the example

AccessibilityManager manager = (AccessibilityManager)this.getSystemService(getApplicationContext().ACCESSIBILITY_SERVICE);

    if(manager.isEnabled()){
        AccessibilityEvent event = AccessibilityEvent.obtain(AccessibilityEvent.TYPE_ANNOUNCEMENT);
        //event.setEventType(AccessibilityEvent.TYPE_ANNOUNCEMENT);
        event.setEnabled(true);
        event.setClassName(getClass().getName());
        event.setPackageName("com.example.App1"); //package name for that third part app
        event.setContentDescription("checkAll:true"); //i used description to send data to accessibility service as i could not find any other option to do so at the time of writing
        //for example you can send the id of button and state of button etc in content description
        event.getText().add("some text");
        manager.sendAccessibilityEvent(event);
        

    }

In onAccessibilityEvent you can capture this event as following

if(accessibilityEvent.getEventType() == AccessibilityEvent.TYPE_ANNOUNCEMENT) {
accessibilityEvent.getContentDescription()//to get content description and then you can parse data out of this string to perform some specific action based on that data}

These are the bits and pieces that i found in last two days. I used these bits and pieces to automate few action on steam app. I know this is not a proper guide also not very accurate maybe. My only purpose of writing this was if someone is in same boat as me, he may get some head start using this for which it took me many days.