Spring Boot: Attempting to override application.properties using EnvironmentPostProcessor

Question

So I'm trying to use key/values stored in Cosul to override values in application.properties. I tried two things.

1) Using Spring Cloud Consul Config. https://cloud.spring.io/spring-cloud-consul/reference/html/#spring-cloud-consul-config

This worked if I did not have the same key defined in my application.properties. If it was defined in application.properties, the value in the properties file was used in all @Value annotation resolution. This is the opposite of what I wanted.

2) Since the above didn't work, I went on to create a custom EnvironmentPostProcessor. I first tried to build a MapPropertySource and used environment.getPropertySources().addAfter(..). This was the same result as above. I then tried to iterate through all property sources, found the one with a name containing "applicationConfig: [classpath:/application" and either set the property value if it exists or putting a new property value. In addition, I added the MapPropertySource to the same EnumerableCompositePropertySource that the "applicationConfig: [classpath:/application" property source is in.

With either approach, it's always the same result. If the key exists in application.properties, that value is used.

What gives? I'm literally overriding the value in the property sources and I can see the values in the debugger before the PostProcessor finishes doing its thing. How is the application.properties value still getting to the @Value annotations?

Here is my current PostProcessor.

@Order(Ordered.LOWEST_PRECEDENCE)
public class ConsulPropertyPostProcessor implements EnvironmentPostProcessor {


    private static final String PROPERTY_SOURCE_NAME = "applicationConfigurationProperties";

    @Override
    public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
        PropertySource<?> system = environment.getPropertySources().get(SYSTEM_ENVIRONMENT_PROPERTY_SOURCE_NAME);

        ConsulKVService consulKVService = new ConsulKVServiceImpl().instantiateConsulKVServiceImpl((String)system.getProperty("CONSUL_HOST"), (String)system.getProperty("CONSUL_TOKEN"));
        Map<String, Object> map = consulKVService.getConsulKeysAndValuesByPrefix((String)system.getProperty("CONSUL_PREFIX"));


        addOrReplace(environment.getPropertySources(), map);
    }

    private void addOrReplace(MutablePropertySources propertySources, Map<String, Object> map) {
        MapPropertySource target = new MapPropertySource("applicationConfig: [consulKVs]", map);
        if (propertySources.contains(PROPERTY_SOURCE_NAME)) {
            PropertySource<?> applicationConfigurationPropertySources = propertySources.get(PROPERTY_SOURCE_NAME);

            for(EnumerableCompositePropertySource applicationPropertySource : (ArrayList<EnumerableCompositePropertySource>)applicationConfigurationPropertySources.getSource()){
                if(applicationPropertySource.getName() != null
                        && applicationPropertySource.getName().contains("applicationConfig: [profile=")) {

                    for(PropertySource singleApplicationPropertySource : applicationPropertySource.getSource()){
                        if(singleApplicationPropertySource.getName().contains("applicationConfig: [classpath:/application")){

                            for (String key : map.keySet()) {
                                if(map.get(key) != null) {
                                    if (singleApplicationPropertySource.containsProperty(key)) {
                                        ((Properties) singleApplicationPropertySource.getSource())
                                                .setProperty(key, (String) map.get(key));
                                    } else {
                                        ((Properties) singleApplicationPropertySource.getSource()).put(key, (String) map.get(key));
                                    }
                                }
                            }
                            break;
                        }
                    }

                    applicationPropertySource.add(target);

                    break;


                }
            }

        }

    }
}

Thanks in advance everyone.

EDIT: Tried overriding the onApplicationEvent method of the ApplicationListener class with the same result as above. Here is that code.

@Log4j
public class ConsulProperties implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {

    static ConfigurableEnvironment configurableEnvironment;
    private static final String PROPERTY_SOURCE_NAME = "applicationConfigurationProperties";

    public static ConfigurableEnvironment getConfigurableEnvironment() {
        return configurableEnvironment;
    }

    @Override
    public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
        log.info("Received ApplicationEnvironmentPreparedEvent...");
        ConfigurableEnvironment environment = event.getEnvironment();
        configurableEnvironment = environment;
        Properties props = new Properties();

        ConsulKVService consulKVService = new ConsulKVServiceImpl()
                .instantiateConsulKVServiceImpl((String) configurableEnvironment.getProperty("CONSUL_HOST"),
                        (String) configurableEnvironment.getProperty("CONSUL_TOKEN"));
        Map<String, Object> map = consulKVService.getConsulKeysAndValuesByPrefix((String) configurableEnvironment.getProperty("CONSUL_PREFIX"));
        while(map.values().remove(null));
        addOrReplace(environment.getPropertySources(), map);
    }


    private void addOrReplace(MutablePropertySources propertySources, Map<String, Object> map) {
        MapPropertySource target = new MapPropertySource("applicationConfig: [consulKVs]", map);
        if (propertySources.contains(PROPERTY_SOURCE_NAME)) {
            PropertySource<?> applicationConfigurationPropertySources = propertySources.get(PROPERTY_SOURCE_NAME);

            for(EnumerableCompositePropertySource applicationPropertySource : (ArrayList<EnumerableCompositePropertySource>)applicationConfigurationPropertySources.getSource()){
                if(applicationPropertySource.getName() != null
                        && applicationPropertySource.getName().contains("applicationConfig: [profile=")) {

                    for(PropertySource singleApplicationPropertySource : applicationPropertySource.getSource()){
                        if(singleApplicationPropertySource.getName().contains("applicationConfig: [classpath:/application")){

                            for (String key : map.keySet()) {
                                if (singleApplicationPropertySource.containsProperty(key)) {
                                    ((Properties) singleApplicationPropertySource.getSource())
                                            .setProperty(key, (String) map.get(key));
                                } else {
                                    ((Properties) singleApplicationPropertySource.getSource()).put(key,
                                            map.get(key));
                                }
                            }


                            applicationPropertySource.add(target);

                            Properties properties = new Properties();
                            properties.putAll(map);
                            propertySources.addLast(new PropertiesPropertySource("consulKVs", properties));

                            break;
                        }
                    }


                    break;


                }
            }
        }
    }
}

Answer 1

It looks like you're trying to change the convention that spring is not designed for. The code you provide is not that easy to maintain and it requires a deep knowledge of Spring internals. Frankly I can't tell without debugging how to achieve what you want, however I have an another approach in mind:

You can use spring profiles in the following way:

Say you have a property db.name=abc in application.properties and db.name=xyz in consul, and I assume your goal is to have db.name=xyz resolved by spring.

In this case, move db.name=abc into application-local.properties and start the application with --spring.profiles.active=local if you want to have the property from the local file, and without this profile if you want to go with consul.

You can even add an active profile dynamically in the EnvironmentPostProcessor (you've got there anyway already), but this is one line of code in EnvironmentPostProcessor.

Answer 2

There is a problem with your code where you are adding the new property source using below code. Please note that when you call "addLast" the property source added through this method takes lowest precedence and will never update the already available properties.

propertySources.addLast(new PropertiesPropertySource("consulKVs", properties));

Instead of above you can use "addFirst" to add the property source which should take highest precedence as given in below code. There are some other methods like "addAfter" & "addBefore" also available which you can explore to add the property source at exact location. In any case "addFirst" will take precedence on all other ways, so I think you can use "addFirst" to update the property source.

propertySources.addFirst(new PropertiesPropertySource("consulKVs", properties));

I have tested this scenario using ApplicationEnvironmentPreparedEvent and it working fine. Hope it should resolve your issue.