Generating JaCoCo coverage report by executing JAR application

Question

I am developing a console Java application (CLI) and trying to write some black-box tests by executing this program, checking output and comparing it with the expected (using JUnit). Everything is ok, but I can't get the code coverage report from JaCoCo.

JaCoCo and JUnit configuration in my build.gradle file:

...

plugins {
    id 'jacoco'
}

...

dependencies {
    ...

    testCompile('org.junit.jupiter:junit-jupiter:5.5.2')
}

test {
    useJUnitPlatform()
}

jacoco {
    toolVersion = "0.8.5"
    reportsDir = file("$buildDir/reports")
}

jacocoTestReport {
    reports {
        xml.enabled true
        csv.enabled false
        xml.destination file("${buildDir}/coverage.xml")
    }

    getExecutionData().setFrom("$buildDir/jacoco/test.exec")
}

In tests I do this:

String command = "java -javaagent:libs/jacocoagent.jar=destfile=build/jacoco/test.exec -jar build/libs/myapp.jar";
Process process = Runtime.getRuntime().exec(command);
process.waitFor();

InputStream inputStream = process.getInputStream();

byte[] b = new byte[inputStream.available()];

String cliOutput = new String(b);

After Test executing files build/jacoco/test.exec and build/coverage.xml are created, but doesn't contain any coverage information.

For simple Unit tests coverage is generating, but it not including coverage from executing the entire program as above.

UPD: Also I tried another way to run JAR:

List<String> args = new ArrayList<>();

args.add("java");
args.add("-javaagent:libs/jacocoagent.jar=destfile=build/jacoco/test.exec");
args.add("-jar");
args.add(this.getCLiPath());
args.add(command);

ProcessBuilder builder = new ProcessBuilder(args.toArray(new String[0]));
Process process = builder.start();

But it didn't help, the same result - no coverage from executing JAR.

UPD 2: I just discovered that execution data is collecting into .exec file.

But this data is being skipped during report generation (in Gradle task jacocoTestReport ) with error messages [ant:jacocoReport] Execution data for class ... does not match. .

It is my execution data from jar file but for some reason, it skipped in the report.

Answer 1

Finally, I've found the solution, so I need to document this.

The error was related to classID. This is a concept described in detail at JaCoCo docs-site - Class Ids . In compiled application class Ids are different and JaCoCo couldn't handle it.

More details in this question . This answer helped me with ant:jacocoReport errors.

Also, I was needed to fix my CI/CD. I am using Azure DevOps Pipelines.

My Gradle task in pipeline:

- task: Gradle@2
    displayName: 'Gradle build, tests, code coverage report'
    inputs:
      gradleWrapperFile: 'gradlew'
      gradleOptions: '-Xmx3072m'
      javaHomeOption: 'JDKVersion'
      jdkVersionOption: '1.8'
      jdkArchitectureOption: 'x64'
      publishJUnitResults: true
      testResultsFiles: '**/TEST-*.xml'
      tasks: 'doJacocoOfflineInstrumentation buildCliJar test jacocoTestReport'