Odd Test Case Failure in IntelliJ / Springboot when Running Entire Suite - Succeeds by Itself

Question

When I run my test case suite in IntelliJ, I have a single junit test case that fails.. The same test case succeeds when run standalone, or with the other tests in the class. It even runs successfully with other tests in other classes. However, when I run the entire suite of 651 tests, this 1 test fails .

Additionally, when I run the test case suite in Maven (ie - maven clean, install), the entire suite runs successfully.

A workaround, which seems to be working, is to set the Fork Mode to "Class" in the run configurations.

However, fork configuration causes the test suite to run slower, and I'd like to understand why this is occurring. The test that fails is below...

@ExtendWith(SpringExtension.class)
@SpringBootTest(classes=MyApplication.class)
class AssetRepositoryTest {

    ....other tests....

    @Transactional
    @Test @DirtiesContext
    void delete_allByAppUser() {
        int deleteCount = assetRepository.deleteAllByAppUser(mainTestAppUser);
        assertEquals(2, deleteCount);
    }

    ....more tests....
}

When the test executes, Springboot spins up, the H2 database is created with some data, and then when the test ends, the database is dropped. The test is meant to test the JPA repository class. It's been working fine until recently and I haven't changed it in some time.

When I run this test case with the entire suite, (the suite seems to always runs in the same order), it fails.

The failure suggests that the Table does not exist...

When I look at the console log, I can see the prior test execute, then spring drops the database. Then this test executes and fails (the db is dropped afterall). And then Spring boots up and JPA creates the database for the next test in the same class. If I didn't know better, I'd say there's a bug with Springboot. It looks like Spring fails to spin up for this single test .

I thought maybe there's an issue with the test class that runs before it, but When I run this test with in a smaller block of tests (same order of execution), including the tests that run before and after it, it runs fine..

And as I said, when I run this standalone, it's fine - spring does it's thing, the database is created, the test executes, and the db is dropped...

Any advice is appreciated. I am unable to determine a pattern or reason for the strange behavior.

Answer 1

The problem I had appeared to be associated with a combination of @DirtiesContext and my Controller Tests. I was using @DirtiesContext on any test that made changes to the H2 database so subsequent tests would not be affected. I wound up removing @DirtiesContext, and for each test that needed database data, I populated it, and then cleaned it up using @SQL annotations (BEFORE and AFTER execution phase). Not only did the problem I have go away, but the 651 tests run MUCH MUCH (significantly) faster.

An example controller test without @DirtiesContext would look like:

@Test
@DisplayName("whitelist remote address - good")
@Sql(scripts={"/sql/remote-addresses.sql"}, executionPhase=Sql.ExecutionPhase.BEFORE_TEST_METHOD)
@Sql(scripts={"/sql/delete-all.sql"}, executionPhase=Sql.ExecutionPhase.AFTER_TEST_METHOD)
void whitelistAddress_good() {
    //Setup app user for test:
    AppUser testUser = TestObjects.getTestUser1();

    //Encrypt data for call:
    String dataToEncode = testUser.getAppUserId() + AppConstants.SPECIAL_DATA_DELIMETER + "1";  //row Id as inserted by the SQL...
    String dataEncoded = encryptionUtil.encrypt(dataToEncode);

    //Setup url variables:
    Map<String, String> variables = new HashMap<>();
    variables.put("encodedData", dataEncoded);

    //Setup headers:
    HttpHeaders headers = new HttpHeaders();
    headers.add("Content-Type", "application/json");

    //Perform call:
    ResponseEntity<String> response = testRestTemplate.exchange(
            unirestTestUtil.getHostUrl() + AppConstants.AUTHORIZE_ADDRESS + "/{encodedData}",
            HttpMethod.PATCH,
            new HttpEntity<>(headers),
            String.class,
            variables
    );

    //Check output status:
    System.out.println("jsonResponse = " + response.getBody());
    assertEquals(HttpStatus.OK.value(),  response.getStatusCodeValue());
}