Some Junit Test cases are failing when running all test classes together but passing individually

Question

Some Junit Test cases are failing when running all test classes together but passing individually.

I am running maven 'mvn clean install' and I could see the following things.

1. One Junit test class (Test1.java) is failing but all other test classes are passing. But same failed test class Test1.java is passing if I run it separately. In both Eclipse (IDE) and command line I noticed the same.
2. If I commented out everything inside one test class (Test2.java) then Test1.java is working when I run all test cases together.

So Test1.java is failing because of Test2.java. But not sure what is the exact cause. I would appreciate if anybody can suggest the root cause. Thanks.

Junit Test1.java:

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class Test1 {

        public static final String REST_API_URI = "/api/someurl/abc";

        @Autowired
        private TestRestTemplate testRestTemplate;
    
        private static final String SVC_QUAL_CHANGE_TOPIC = "tnco-svcqual-change";
    
        @ClassRule
        public static EmbeddedKafkaRule embeddedKafka = new EmbeddedKafkaRule(1,
                true,
                1,
                SVC_QUAL_CHANGE_TOPIC);
        
        @BeforeClass
        public static void setUpBeforeClass() throws IOException {
            System.setProperty("spring.kafka.bootstrap-servers", embeddedKafka.getEmbeddedKafka().getBrokersAsString());
        }
    
        @AfterClass
        public static void tearDownAfterClass() {
            System.clearProperty("spring.kafka.bootstrap-servers");
        }
    
        @Test
        public void testCreateCheckServiceQualification() {
          ...
          //some API test code which send msg to Kafka
        }   
    }

Junit Test2.java:

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = { "alm.ishtar.security.enabled = false" })
public class Test2 {

    public static final String REST_API_URI = "/api/someurl/xyz";

    @Autowired
    private TestRestTemplate testRestTemplate;
    
    private static final String MOI_CHANGE_TOPIC = "tnco-moi-change";
    
    @ClassRule
    public static EmbeddedKafkaRule embeddedKafka = new EmbeddedKafkaRule(1,
            true,
            1,
            MOI_CHANGE_TOPIC);
    
    @BeforeClass
    public static void setUpBeforeClass() throws IOException {
        System.setProperty("spring.kafka.bootstrap-servers", embeddedKafka.getEmbeddedKafka().getBrokersAsString());
    }

    @AfterClass
    public static void tearDownAfterClass() {
        System.clearProperty("spring.kafka.bootstrap-servers");
    }

    @Test
    public void testCreateMoiPut() throws Exception {
      ...
      //some API test code which send msg to Kafka
    }
}

Getting the below error in the console at the time of test failure:

org.springframework.kafka.KafkaException: Send failed; nested exception is org.apache.kafka.common.errors.TimeoutException: Topic tnco-svcqual-change not present in metadata after 60000 ms.
org.springframework.kafka.KafkaException: Send failed; nested exception is org.apache.kafka.common.errors.TimeoutException: Topic tnco-svcqual-change not present in metadata after 60000 ms.
        at org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:573)
        at org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:401)

Note: Test method calls REST API and inside the API implementation it is sending message to Kafka topic.

Answer 1

From the error you provided, it is obvious that you run tests in parallel .

And this is a shared resource : System.setProperty("spring.kafka.bootstrap-servers", embeddedKafka.getEmbeddedKafka().getBrokersAsString());

I can assume that

1. Test1 gets executed, it creates and configures embeddedKafka with SVC_QUAL_CHANGE_TOPIC , and sets its address to the shared property spring.kafka.bootstrap-servers .
2. Then Test2 gets executed, configures another embeddedKafka with MOI_CHANGE_TOPIC and now property spring.kafka.bootstrap-servers points to the second embeddedKafka.
3. Test method from Test1 is executed, reads property spring.kafka.bootstrap-servers and uses "wrong" embeddedKafka.

Solution would be to

1. Exclude tests which rely on the same resource (system property spring.kafka.bootstrap-servers in this case) from the parallel run. And run them consequentially. Like here Exclude specific tests from being run in parallel in jUnit
2. Combine both tests in one and configure embeddedKafka with both topics.

Note, that setting up embeddedKafka before each test (instead of before Class) won't help, as nothing prevents Test1.method1 from executing at the same time Test2.methodX changes the property.