如何设置Jenkins运行android单元测试？

Question

I have recently installed Jenkins on my development machine to help me learn how continuous integration/delivery works.

I have a basic Android task setup that will build my app and then report lint, PMD, Checkstyle, Findbugs, and JUnit (business logic) test results.

I would also like it to run android unit tests (those that require an emulator to run), how does one set this up? After I have added 'connectedAndroidTest' to the list of gradle commands to run during the build I get the following output:

17:03:21 :app:connectedDebugAndroidTest FAILED
17:03:21 
17:03:21 FAILURE: Build failed with an exception.
17:03:21 
17:03:21 * What went wrong:
17:03:21 Execution failed for task ':app:connectedDebugAndroidTest'.
17:03:21 > com.android.builder.testing.api.DeviceException: No connected devices!
17:03:21 
17:03:21 * Try:
17:03:21 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
17:03:21 
17:03:21 BUILD FAILED

This is expected, so I tried starting an emulator from Android Studio and running the Jenkins build again, resulting in this error:

17:08:54 :app:connectedDebugAndroidTest
17:08:54 05:08:54 E/SplitApkInstaller: Failed to finalize session : INSTALL_FAILED_UPDATE_INCOMPATIBLE: Package com.zao.testapp.test signatures do not match the previously installed version; ignoring!
17:08:54 Unable to install C:\Program Files (x86)\Jenkins\workspace\testapp\app\build\outputs\apk\app-debug-androidTest.apk
17:08:54 com.android.ddmlib.InstallException: Failed to finalize session : INSTALL_FAILED_UPDATE_INCOMPATIBLE: Package com.zao.testapp.test signatures do not match the previously installed version; ignoring!
17:08:54    at com.android.ddmlib.SplitApkInstaller.install(SplitApkInstaller.java:92)
17:08:54    at com.android.ddmlib.Device.installPackages(Device.java:904)
17:08:54    at com.android.builder.testing.ConnectedDevice.installPackages(ConnectedDevice.java:136)
17:08:54    at com.android.builder.internal.testing.SimpleTestCallable.call(SimpleTestCallable.java:134)
17:08:54    at com.android.builder.internal.testing.SimpleTestCallable.call(SimpleTestCallable.java:49)
17:08:54    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
17:08:54    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
17:08:54    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
17:08:54    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
17:08:54    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
17:08:54    at java.lang.Thread.run(Thread.java:748)
17:08:54 
17:08:54 com.android.builder.testing.ConnectedDevice > runTests[emulator-5554 - 8.0.0] [31mFAILED [0m
17:08:54    com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException: Failed to finalize session : INSTALL_FAILED_UPDATE_INCOMPATIBLE: Package com.zao.testapp.test signatures do not match the previously installed version; ignoring!
17:08:54        at com.android.builder.testing.ConnectedDevice.installPackages(ConnectedDevice.java:144)

17:08:54 [no message defined]
17:08:54 com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException: Failed to finalize session : INSTALL_FAILED_UPDATE_INCOMPATIBLE: Package com.zao.testapp.test signatures do not match the previously installed version; ignoring!
17:08:54    at com.android.builder.testing.ConnectedDevice.installPackages(ConnectedDevice.java:144)
17:08:54    at com.android.builder.internal.testing.SimpleTestCallable.call(SimpleTestCallable.java:134)
17:08:54    at com.android.builder.internal.testing.SimpleTestCallable.call(SimpleTestCallable.java:49)
17:08:54    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
17:08:54    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
17:08:54    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
17:08:54    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
17:08:54    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
17:08:54    at java.lang.Thread.run(Thread.java:748)
17:08:54 Caused by: com.android.ddmlib.InstallException: Failed to finalize session : INSTALL_FAILED_UPDATE_INCOMPATIBLE: Package com.zao.testapp.test signatures do not match the previously installed version; ignoring!
17:08:54    at com.android.ddmlib.SplitApkInstaller.install(SplitApkInstaller.java:92)
17:08:54    at com.android.ddmlib.Device.installPackages(Device.java:904)
17:08:54    at com.android.builder.testing.ConnectedDevice.installPackages(ConnectedDevice.java:136)
17:08:54    ... 8 more
17:08:54 :app:connectedDebugAndroidTest FAILED
17:08:54 
17:08:54 FAILURE: Build failed with an exception.
17:08:54 
17:08:54 * What went wrong:
17:08:54 Execution failed for task ':app:connectedDebugAndroidTest'.
17:08:54 > There were failing tests. See the report at: file:///C:/Program%20Files%20(x86)/Jenkins/workspace/testapp/app/build/reports/androidTests/connected/index.html
17:08:54 
17:08:54 * Try:
17:08:54 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
17:08:54 
17:08:54 BUILD FAILED

I also tried setting up an emulator through the task setup. There is an option to "Run an Android emulator during build" that I have checked.

The option to run an existing emulator doesn't work for me - I give it the AVD name from the emulator I created in Android Studio, in my case "Nexus_5X_API_26" and I get the following error when I build:

17:26:07 FATAL: C:\Windows\system32\config\systemprofile\.android\avd\Nexus_5X_API_26.ini (The system cannot find the path specified)
17:26:07 java.io.FileNotFoundException: C:\Windows\system32\config\systemprofile\.android\avd\Nexus_5X_API_26.ini (The system cannot find the path specified)

Which makes sense since my emulator is in C:\\Users\\Zach.android\\avd\\ .

I also tried to run an emulator with properties that I specify in the config, using the following values:

Android OS version: android-26
Screen density: 420
Screen resolution: 1080x1920
Device locale: en_US
SD card size: <empty>
Target ABI: x86
Emulator name suffix: <empty>

And I get the following output:

17:31:51 $ D:\Android\sdk/tools/android.bat list target
17:31:52 [android] The configured Android platform needs to be installed: android-26
17:31:52 $ D:\Android\sdk/tools/android.bat list target
17:31:53 $ D:\Android\sdk/tools/android.bat list target
17:31:54 [android] Installing the 'sys-img-x86-android-26' SDK component(s)...
17:31:54 $ D:\Android\sdk/tools/android.bat update sdk -u -a -t sys-img-x86-android-26
17:31:54 **************************************************************************
17:31:54 The "android" command is deprecated.
17:31:54 For manual SDK, AVD, and project management, please use Android Studio.
17:31:54 For command-line tools, use tools\bin\sdkmanager.bat
17:31:54 and tools\bin\avdmanager.bat
17:31:54 **************************************************************************
17:31:54 
17:31:54 "android" SDK commands can be translated to sdkmanager commands on a best-effort basis.
17:31:54 (This prompt can be suppressed with the --use-sdk-wrapper commend-line argument
17:31:54 or by setting the USE_SDK_WRAPPER environment variable)

Which causes Jenkins to hang forever with a busy indicator at the last line of the console output view. I am forced to abort the build and restart Jenkins at this point in order to have builds work again.

So, what am I missing here? Why is this so difficult for me! Anyone know how to accomplish running Android unit tests -> get a simulator to kick off for a build on Jenkins?

EDIT : I updated Jenkins to run as 'me' rather than the local system account and that allowed me to (try to) use an existing emulator rather than complain about not being able to find it. However, I now get this error when trying to start the emulator:

00:28:11 $ D:\Android\sdk/tools/android.bat list target
00:28:12 [android] Using Android SDK: D:\Android\sdk
00:28:12 $ D:\Android\sdk/platform-tools/adb.exe start-server
00:28:16 * daemon not running. starting it now at tcp:5847 *
00:28:16 * daemon started successfully *
00:28:16 $ D:\Android\sdk/platform-tools/adb.exe start-server
00:28:16 [android] Starting Android emulator
00:28:16 $ D:\Android\sdk/tools/emulator.exe -engine classic -ports 5688,5689 -report-console tcp:5831,max=60 -avd Nexus_5X_API_26 -no-snapshot-load -no-snapshot-save -no-window
00:28:16 PANIC: Missing emulator engine program for 'x86' CPU.
00:29:21 [android] Emulator did not appear to start; giving up
00:29:21 [android] Stopping Android emulator
00:29:22 $ D:\Android\sdk/platform-tools/adb.exe kill-server

So I guess this is the number one question for now, any thoughts?

Answer 1

I was able to get this working by adding the "-engine auto" emulator option.

I looked at this Jenkins issue to help solve this: https://issues.jenkins-ci.org/browse/JENKINS-43557