SonarQube Test Custom Java Rule fails to display semantic information

Question

I am learning the API of SonarQube, trying to extend the java plugin rules. I followed this tutorial successfully.

Now I want to build a simple analysis that checks if a toString() method is used in a unit test.

public class TS_SensitiveEqualityCheck extends BaseTreeVisitor implements JavaFileScanner {

  private final Deque<Boolean> methodContainsToStringInAssert = new ArrayDeque<Boolean>();
  private final Deque<Boolean> inUnitTest = new ArrayDeque<Boolean>();

  private JavaFileScannerContext context;

  @Override
  public void scanFile(final JavaFileScannerContext context) {
    this.context = context;
    scan(context.getTree());
  }

  @Override
  public void visitMethod(MethodTree methodTree) {
    if (ModifiersUtils.hasModifier(methodTree.modifiers(), Modifier.ABSTRACT)) {
      return;
    }
    boolean isUnitTest = isUnitTest(methodTree);
    inUnitTest.push(isUnitTest);
    System.out.println("For method " + methodTree.simpleName() 
    + " found [isUnitTest | isViolation] :  " 
    + String.valueOf(isUnitTest));
    methodContainsToStringInAssert.push(false);
    super.visitMethod(methodTree);
    inUnitTest.pop();
    Boolean isViolation = methodContainsToStringInAssert.pop();
    System.out.println("For method " + methodTree.simpleName() 
            + " found [isUnitTest | isViolation] :  " 
            + String.valueOf(isUnitTest) + " " 
            + String.valueOf(isViolation) );
    if (isUnitTest && isViolation) { 
      context.reportIssue(this, methodTree.simpleName(), "This test method uses unsafe equality checking!");
    }
  }

  @Override
  public void visitMethodInvocation(MethodInvocationTree mit) {
    if (!inUnitTest()) {
      return;
    }
    Symbol mis = mit.symbol();
    System.out.println(mis.name()); // null when encountering an assertion.
    if (mis.name() != null && mis.name().equals("toString")) {
        setTrue(methodContainsToStringInAssert);
    }
    super.visitMethodInvocation(mit);
  }

  private boolean inUnitTest() {
    return !inUnitTest.isEmpty() && inUnitTest.peek();
  }

  private static void setTrue(Deque<Boolean> collection) {
    if (collection != null && !collection.peek()) {
      collection.pop();
      collection.push(true);
    }
  }

  private static boolean isUnitTest(MethodTree methodTree) {
    JavaSymbol.MethodJavaSymbol symbol = (JavaSymbol.MethodJavaSymbol) methodTree.symbol();
    while (symbol != null) {
      if (symbol.metadata().isAnnotatedWith("org.junit.Test")) {
        return true;
      }
      symbol = symbol.overriddenSymbol();
    }
    Symbol.TypeSymbol enclosingClass = methodTree.symbol().enclosingClass();
    return (enclosingClass != null 
            // && enclosingClass.type().isSubtypeOf("junit.framework.TestCase")  // errors!!! does not get the package name of the class!!!
            && methodTree.simpleName().name().startsWith("test"));
  }

}

For the given test file SonarQube cannot find any assertion method invocation! Only the method B.is() gives a result, ie mit.symbol().name != null . Can anyone explain why this goes wrong? Here is the file that is used as test:

import junit.framework.TestCase;
    import javax.annotation.Nullable;


public class AssertionsInTestsCheckTestJunit3 extends TestCase {

public void testCompliant() {
    B b = new B();
    b.is();
  org.junit.Assert.assertTrue(b.is());
}

public void testNoncompliant() { // Noncompliant
    org.junit.Assert.assertTrue(this.toString().equals(""));
}

public void testNoncompliant2() { // Noncompliant
    org.junit.Assert.assertEquals(this.toString(), "");
}

public void testNoncompliant3() { // Noncompliant
    org.junit.Fail.fail(this.toString());
    doWork();
}

@Nullable
public Test notAtest() {
  compliant1();
}

}

public class B {
   public boolean is() {
      return true;
  }
} </pre></code>

Note that it is not important what this code does!

Answer 1

The Java Analyzer requires the byte code of the libraries used in source files in order to complete the semantic model. Without it, most of the semantic information which could be retrieved is missing.

In your test file, you are using junit . However, binaries related to junit are missing, as you are most probably not providing the library to the check verifier. This part has not been described in the tutorial yet.

By default, no external libraries are provided to the check verifier, as the tutorial, in its current state, doesn't requires external libraries. The part explaining how to use external sources has not yet been written, but once done, it should be available following this link: How to test sources requiring external binaries .

Now, to solve your issue and sum up what is going to be presented in the tutorial, here is what you have to do:

In order to use the adequate byte code when analyzing the file, you have to provide the junit binaries to the check verifier. There is multiple approach to do so, but the simplest one is probably to provide the jar by placing it at a dedicated location in your project: target/test-jars . By default, the check verifier is going to look in this folder.

You can add any required library automatically, by changing the pom.xml file at the root of the project:

1. In the pom.xml file at the root of your plugin, you should see a commented part: pom.xml#L108 (from the template used in the tutorial)
2. Un-comment this part of code. Note that for the example, it already contains the binaries of apache commons-collections.

 <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>2.10</version>
    <executions>
      <execution>
        <id>copy</id>
        <phase>test-compile</phase>
        <goals>
          <goal>copy</goal>
        </goals>
        <configuration>
          <artifactItems>
            <artifactItem>
              <groupId>org.apache.commons</groupId>
              <artifactId>commons-collections4</artifactId>
              <version>4.0</version>
              <type>jar</type>
            </artifactItem>
          </artifactItems>
          <outputDirectory>${project.build.directory}/test-jars</outputDirectory>
        </configuration>
      </execution>
    </executions>
  </plugin>

3. Replace the <artifactItem> to use junit instead, or simply add a new artifact:

  <artifactItem>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <type>jar</type>
  </artifactItem>

4. Re-build the project with mvn clean install -DskipTests . The junit jar with the required version will be downloaded by maven, and placed in your target/test-jars folder.

5. Re-run your test. Issues should be detected.