Test Class with @RunWith(PowerMockRunner.class) annotation runs test cases twice when extending another class

Question

I am working on unit tests. Our project is working with the Play! framework. The unit tests are written in Java. Our team has encountered a problem where we have been experiencing duplicate test runs for test classes which are annotated with @RunWith(PowerMockRunner.class) and extend another class.

Our main set up for test classes are as follows:

A Unit Test Harness: Contains functionality shared among various test cases. Also contains commonly used fields.

@Ignore
@RunWith(PowerMockRunner.class)
@PrepareForTest(SomeStaticClass.class)
public abstract class UnitTestHarness {
    //Bunch of setup code...
}

A Test Class: Contains the test cases for the class we're testing.

public class TestClass extends UnitTestHarness {
    @Test
    public void testSomething(){
        //Perform some tests...
    }
}

As you can see, TestClass extends UnitTestHarness which has the annotation. When TestClass is run by the play "test-only" command, the following output is returned:

[info] somepackage.TestClass
[info] + testSomething
[info] 
[info] 
[info] Total for test somepackage.TestClass
[info] Finished in 0.055 seconds
[info] 1 tests, 0 failures, 0 errors
[info] somepackage.TestClass
[info] + testSomething
[info] 
[info] 
[info] Total for test somepackage.TestClass
[info] Finished in 0.001 seconds
[info] 4 tests, 0 failures, 0 errors
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] Total time: 18 s, completed Apr 16, 2014 4:19:37 PM

Obviously, the tests are being run twice for some reason. I've read online that there have been some issues with the @RunWith annotation and extending classes, but I haven't been able to find a solution. I feel it is also import to note that if I move the annotation from UnitTestHarness to TestClass, the following output will be returned:

[info] somepackage.TestClass
[info] 
[info] 
[info] Total for test somepackage.TestClass
[info] Finished in 0.032 seconds
[info] 0 tests, 0 failures, 0 errors
[info] somepackage.TestClass
[info] + testSomething
[info] 
[info] 
[info] Total for test somepackage.TestClass
[info] Finished in 0.025 seconds
[info] 4 tests, 0 failures, 0 errors
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] Total time: 13 s, completed Apr 16, 2014 4:47:28 PM

In this case, we can see that it's still trying to run some sort of tests, but it isn't actually running the additional 4 because the UnitTestHarness does not have the annotation. I'm guessing this is because while in the first example the annotation was placed on the UnitTestHarness and was inherited by TestClass, in this example only the TestClass has the annotation so the tests are only run for that particular class.

Has anyone else seen this issue? Obviously we can make due with the second scenario, but that additional attempt to run those classes shouldn't be happening at all ideally. Is this a problem with the PowerMockRunner or the Play framework? The tests are also run twice when I run a mvn clean install or play test to run the entire test suite. Any possible solutions or suggestions would be greatly appreciated. Thank you all so much for your help.

Suggested Solution #1 : Inject UnitTestHarness

I was able to get them not to run twice by using MockitoAnnotations.initMocks(this); instead of the annotation you mentioned. When you use the annotation, the tests still run twice. I tried to use this method described but ran into a variety of problems both with and without the annotation.

1) This doesn't seem like the proper use of @InjectMocks. I've used that annotation and it's really meant to be used when you have fields mocked out that you want to inject into another object which has a field of that type.

2) I've found that the mock behaviors from the UnitTestHarness, when injected in this manner, don't transfer. Our Unit Test Harness sets up a current Http.Context and this does not transfer to our test case. In fact, a lot of the initialization that made UnitTestHarness useful would need to be transferred into an @Before method in the TestClass itself which defeats the purpose of having the harness in the first place I feel.

3) All methods that we have and use in the harness will have to be made public for accessibility and accessed either by the instance of the harness or static calls in the TestClass itself. We would also need to either make fields in the harness public, or create mutators for a majority of them.

Overall I don't think this is a good solution to the problem we're having, but I appreciate your idea.

Answer 1

Did you try to inject UnitTestHarness into TestClass using annotation @InjectMocks and call all necessary methods in method annotated by @Before? Remeber that when you use @InjectMocks, you also need add above the class name @RunWith(MockitoJUnitRunner.class). Here are some examples.

Avoidance of inheritance is always a good choice.

Edit:

It seems to me that the problem is not in the the library. I wrote some code and it's work fine:

public final class ClassWithStaticMethods {

   public static String returnString(){
       return "Sample text";
   }
}

@RunWith(PowerMockRunner.class)
@PrepareForTest(ClassWithStaticMethods.class)
public class UnitTestHarness {

    private String field1 = "field1";
    private String field2 = "field2";

    @Before
    public void setUp() {
        PowerMockito.mockStatic(ClassWithStaticMethods.class);
        when(ClassWithStaticMethods.returnString()).thenReturn("This static method is mocked!");
    }

    public List<String> createList() {
        List<String> list = Lists.newArrayList();
        list.add("firstElement");
        list.add("secondElement");
        list.add("thirdElement");

        return list;
    }

    public Point dummyPoint() {
        return new Point(2, 3);
    }

    public String getField1() {
        return field1;
    }

    public void setField1(String field1) {
        this.field1 = field1;
    }

    public String getField2() {
        return field2;
   }

    public void setField2(String field2) {
        this.field2 = field2;
    }
}


public class TestClass extends UnitTestHarness {

    private List<String> list;
    private Point dummyPoint;

    @Before
    public void setup() {
        list = createList();
        dummyPoint = dummyPoint();
    }

    @Test
    public void testSomething() {
        System.out.println("First test method. Value of field1 from UnitTestHarness is: " + getField1());
        setField1("New value");
        System.out.println("First test method. Value of field1 from UnitTestHarness is: " + getField1());
    }

    @Test
    public void testSomethingElse() {
        System.out.println("Second test method. Value of field2 from UnitTestHarness is: " + getField2());
    }

    @Test
    public void testStaticMockMethod() {
        System.out.println(ClassWithStaticMethods.returnString());
    }

    @Test
    public void testDummyPoint() {
        System.out.println(String.format("My point: %s, %s", dummyPoint.getX(), dummyPoint.getY()));
    }

    @Test
    public void testArray() {
        for (String string : list) {
            System.out.println("Array element:" + string);
        }
   }
}

This is my console output:

Running service.TestClass
Array element:firstElement
Array element:secondElement
Array element:thirdElement
First test method. Value of field1 from UnitTestHarness is: field1
First test method. Value of field1 from UnitTestHarness is: New value
Second test method. Value of field2 from UnitTestHarness is: field2
This static method is mocked!
My point: 2.0, 3.0
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.442 sec

Results :

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0

Without the broader context of this problem it's hard to advise something more concrete, maybe you should add more code which is inside this classes?