How to configure different Kafka Brokers/endpoints for different test environments?

Question

I am using Kafka in my integration tests wherein it publishes to different topics and my tests reads from them and validates. Now, I have created a class with different Kafka constants/endpoints, but these constants vary on different environments; say for eg - ST, SIT, etc. How do I configure these constants as per environment so that in pipeline whichever environment am running my tests on it picks up the right constants/endpoints.

It currently looks like below, please guide how to configure on various environments.

package Kafka;

//ST

public interface KafkaConst {

public static String KAFKA_BROKERS = "https://10.156.192.120:1211";

public static Integer MESSAGE_COUNT=10;

public static String INBOUND_TOPIC_NAME="publish.st"

public static String GROUP_ID_CONFIG="consumerGroup1";

public static String SCHEMA_REGISTRY = "http://10.156.192.71:1212";

public static Integer MAX_NO_MESSAGE_FOUND_COUNT=10;

public static String OFFSET_RESET_LATEST="latest";

public static String OFFSET_RESET_EARLIER="earliest";

public static Integer MAX_POLL_RECORDS=1000; 

public static String KAFKA_File="src/test/resources/TransformedXML/";

}

//SIT

public interface KafkaConst {

public static String KAFKA_BROKERS = "https://10.156.165.120:1211";

public static Integer MESSAGE_COUNT=10;

public static String INBOUND_TOPIC_NAME="publish.sit"

public static String GROUP_ID_CONFIG="consumerGroup1";

public static String SCHEMA_REGISTRY = "http://10.156.165.71:1212";

public static Integer MAX_NO_MESSAGE_FOUND_COUNT=10;

public static String OFFSET_RESET_LATEST="latest";

public static String OFFSET_RESET_EARLIER="earliest";

public static Integer MAX_POLL_RECORDS=1000; 

public static String KAFKA_File="src/test/resources/TransformedXML/";

}

Answer 1

I would suggest getting away from hardcoded values and use environment variables. Most of the CI/CD tools support injecting environment variables in the pipeline runtime, while in your code you take those values not from hardcoded classes but from environment variables.

I see that you've added the spring-kafka tag... in a Spring app, it's super easy to get values from environment variables. You can either use the @Value annotation or the @ConfigurationProperties one. There are tons of examples on the internet for both.

Answer 2

Use Spring Profiles to select which properties apply in each environment.

It is often useful to conditionally enable or disable a complete @Configuration class or even individual @Bean methods , based on some arbitrary system state. One common example of this is to use the @Profile annotation to activate beans only when a specific profile has been enabled in the Spring Environment (see Bean Definition Profiles for details).