Java Mockito one test causes another to fail

Question

@Test
    public void onConnectionCompletedTest() {
        connectionProvider.initialize();
        connectionProvider.addEventObserver(SocketEvent.Type.SOCKET_CONNECT, mockedObserver);
        connectionProvider.onConnectionCompleted(mockedChannel);

        verify(mockedObserver).socketEventObserved(socketEventCaptor.capture());
        Assert.assertEquals(SocketEvent.Type.SOCKET_CONNECT, socketEventCaptor.getValue().getType());
    }


@Test
    public void onConnectionClosedTest() {
        connectionProvider.initialize();
        connectionProvider.addEventObserver(SocketEvent.Type.SOCKET_DISCONNECT, mockedObserver);
        connectionProvider.onConnectionClosed(mockedChannel);

        verify(mockedObserver).socketEventObserved(socketEventCaptor.capture());
        Assert.assertEquals(SocketEvent.Type.SOCKET_DISCONNECT, socketEventCaptor.getValue().getType());
    }

The problem is when I run both of these tests, the 2nd one fails. But if I comment out

verify(mockedObserver).socketEventObserved(socketEventCaptor.capture());
        Assert.assertEquals(SocketEvent.Type.SOCKET_CONNECT, socketEventCaptor.getValue().getType());

then the 2nd test will pass. There's a lot of different classes/methods involved in this so hopefully this is enough information to be able to come up with an explanation.

The error I get:

wanted but not invoked:
mockedObserver.socketEventObserved(
    <Capturing argument>
);
-> at com.company.cnx.cip.io.ConnectionProviderTest.onConnectionClosedTest(ConnectionProviderTest.java:188)
Actually, there were zero interactions with this mock.

My question exactly: What could be happening that when I @Ignore the first test, the 2nd will pass?

EDIT: I have an @Before class that's important.

@Before
    public void init() {
        MockitoAnnotations.initMocks(this);
        JsonParser parser = new JsonParser();
        JsonElement jsonElement = parser.parse(json);
        configurationService.loadConfiguration(jsonElement, "id");
        AppContext.getContext().applyConfiguration(configurationService);
        connectionProvider = ConnectionProvider.newInstance();

    }

You can ignore everything that isn't the first and last line of it. I make a new ConnectionProvider object, so I would think that one test shoudldn't affect another, because they're operating on two separate objects.

Answer 1

This is more straightforward than you think: The test works when you comment out those lines because socketEventObserved is not getting called when it's the second test of the run. This is probably a problem with code that you haven't posted above.

Since it seems that the culprit may be buried in an impractical volume of code, here are a few debugging tips and general sources of test pollution:

• First and foremost, set a breakpoint everywhere that socketEventObserved is called, and compare between single-test runs and multi-test runs. If you see the same behavior Mockito sees, then it's not Mockito.

• As it turned out to be in this case, keep an eye out for actions that may occur on other threads (particularly to listeners). Using a Mockito timeout can help there:

   verify(mockedObserver, timeout(2000)) .socketEventObserved(socketEventCaptor.capture()); 

• You seem to be working with I/O channels, which sometimes involve buffering or flushing policies that can be triggered only when a certain number of tests are run, or if tests are run in certain order. Make sure that your @Before method fully resets state, including any modes or buffers that your code may touch.

• You interact with AppContext.getInstance() and ConnectionProvider.newInstance() , both of which are static method calls. I'm less worried about the latter, unless it conserves instances in spite of its name, but the former may not take kindly to multiple initializations. In general, in your system-under-test, keep an eye out for writes to global state.

• Mockito itself keeps static state. Mockito keeps its internal state static and thread-scoped (through ThreadLocal), and at times a test can leave Mockito in an invalid internal state that it cannot detect (because an operation is half-completed, for instance). Add an @After method to detect this case:

   @After public void checkMockito() { Mockito.validateMockitoUsage(); } 

  ...or switch to using MockitoRule or MockitoJUnitRunner , which do this and initMocks(this) automatically.

• Finally, Mockito isn't perfect: It uses proxy objects to override methods, which also means it will silently fail to mock final methods and some non- public methods that work with nested classes (as those require the compiler to generate synthetic methods you can't see). If your mockedObserver has any final or limited-visibility methods, it may cause real code and mocked code to interact in a way that makes the system's behavior hard-to-predict.