HTML中如何自动生成Android测试报告

Question

I would like to automatically generate unit testing report in html format for Android application on Hudson continuous integration server.

Therefore, I try to run test cases first and gather test result files in xml format. Then, I use JUnitReport Task to transform the XML result files into HTML format.

I run test cases through Android instrumentation framework. However, it only provides verbose output information rather than the standard JUnit XML format. I have no idea how to generate HTML unit test report without JUnit XML result files.

If I run test cases using Eclipse, it can export results in XML files with time consumed information per test case. Those XML files can be transformed into HTML by JUnitReport Task correctly. As a result, it seems that it is possible to collect the test result with time consumed information.

Is there any way to get the standard JUnit XML result file automatically after running test cases on Android instrumentation framework?

Answer 1

We had similar problem in our company. We checked all the available open-source solutions and none of them was really perfect. So we developed and just open-sourced a solution for it. I still do not say an "ultimate" one but certainly much better than either athena or the python reporter or any after-test analysis. You can find it here: http://code.google.com/p/the-missing-android-xml-junit-test-runner/

It provides:

• separate XML file per each package involved
• XML files are generated on the device (need to be adb pull'ed after test)
• timing of the tests is fully supported
• we have full stack trace reported in failure/error case

Instead of analysing java source code (as in athena) or analysing the output (the python script), we extended android instrumentation runner. So we get all the benefits of using standard command line options for test selection, coverage enabling etc. - all described here: http://developer.android.com/guide/developing/testing/testing_otheride.html#RunTestsCommand .

We were able to successfully run the code using standard test rules with coverage analysed by emma, all nicely reported in Jenkins.

Answer 2

I don't think it is possible, so you need to create a utility that converts the Android test runner output into JUnit-format XML files.

However, you won't be able to get the time-per-test value, as the Android test runner doesn't seem to output that information.

Answer 3

I think the athena approach is the lest painfull as you can simply install the athena jar file on your system and add following ant target

<property name="junit.dir" value="${basedir}/junit-results"/>
<property name="athena.jar" value="/home/cruise/athena/athena-1.1.2.jar"/>

<target name="prepare" description="Setup needed directories">
  <mkdir dir="${junit.dir}"/>
</target>

<!-- This target will compile/install tested project as well as test project to ensure tests are executed against latest code -->
<target name="athena" depends="prepare, -install-tested-project, install" description="Run tests and convert result to xml using athena">
  <exec executable="java" os="Linux" failonerror="true">
    <arg value="-cp"/>
    <arg value="${athena.jar}"/>
    <arg value="com.synaptik.athena.Athena"/>
    <arg value="${basedir}"/>
    <arg value="${junit.dir}/TEST-result.xml"/>
  </exec>
</target>

From command line it is then simply a matter of running ant athena and you have the test result in xml. This will also be the target your CruiseControl script should trigger.

Answer 4

I'm in a similar situation and found https://docs.marathonlabs.io/ works great for my use case. Other options I've looked into:

• https://docs.marathonlabs.io/ is an effort similar to spoon but seems to be more updated and better maintained. It also provides nice HTML reports with additional features like tests batching, retrying and screen recording.
• https://github.com/square/spoon is a test runner wrapper on top of adb shell am instrument which first runs with -e log true to get a list of available Espresso tests, and then run each test case one-by-one with -r flag so that we can be getting the XML test result from the raw text output as well runtime at the per testcase level. I didn't go down this route because it makes our overall test execution time a lot longer.
• Use https://github.com/jamesknowsbest/Instrumentationpretty to parse the output from instrumentation tests raw log (-r) flag. We will be able to generate an XML JUnit test result with all info, except for test execution time at the test case level.
• Use adb shell am instrument -f <proto_file> to store the instrumentation result in a protobuf file, read it in and convert it to junit xml file. Looking at the Android instrument source code for am/instrument.java , it seems the testlevel runtime are kept when we run it with -f instead of -r. But this means we will have to create our own protobuf parser file to generate the Junit-style XML file yourself.
• https://github.com/schroepf/TestLab/tree/master/android is another Open source tool created to add instrumentation data inside the application (as opposed to the client / runner side approach above). It should create a JUnit style XML result inside the Android emulator and we can then use adb pull to pull it out of the emulator afterwards.