一段时间后，生产春季启动应用程序将引发NoSuchBeanDefinitionException

Question

We had a medium complicated spring boot 1.5.14 app with rest api + mybatis for backend, angular 4 with material/prime-ng for frontend. It works fine from developers' box up to UAT environments, but in production, it works fine for the first couple of days, then throws NoSuchBeanDefinition. The production environment is openshift + openjdk version "1.8.0_171".

To trim down the app and leave related info, here are code snippets:

public interface ITaxCalculator {
    BigDecimal calc(BigDecimal amount);
}

public class FedProvTaxCalculator implements ITaxCalculator {
    ... ...
}

@Configuration
public class TaxCalculatorConfiguration {
    ...
    @Bean("onTaxCalculator")
    public ITaxCalculator ontairioTaxCalculator() {
        FedProvTaxCalculator ret = ..
        ...
        return ret;
    }

    @Bean("bcTaxCalculator")
    public ITaxCalculator britishColumbiaTaxCalculator() {
        FedProvTaxCalculator ret = ..
        ...
        return ret;
    }

}

public class CAOrderProcessor implements IOrderProcessor {
    @Autowire @Qualifier("onTaxCalculator")
    private FedProvTaxCalculator onTaxCalculator;

    @Autowire @Qualifier("bcTaxCalculator")
    private FedProvTaxCalculator bcTaxCalculator;

    ....
}

// --------------- below code are at framework level -----

public interface IOrderProcessor {
    void process(Order order);
}

public interface IOrderProcessorFactory {
    IOrderProcessor createOrderProcessor(String countryCode, MembershipType membership);
}

@Service
public class OrderProcessorFactoryPropImpl implements IOrderProcessorFactory {
    @Autowired
    private AutowireCapableBeanFactory beanFactory;

    @Override
    @Cacheable("orderProcessor")
    public IOrderProcessor createOrderProcessor(String countryCode, MembershipType membership) {
        String clzName = resolveOrderProcessClzName(countryCode, membership);       // resolve to CAOrderProcess clz-name
        try {
            Object ret = Class.forName(clzName).newInstance();
            beanFactory.autowireBean(ret);          
            // the above line throws error after a while 
            return (IOrderProcessor)ret;
        } catch (Exception ex) {
          ...
          throw new RuntimeException(...);
        }
    }

    private String resolveOrderProcessClzName(String countryCode, MembershipType membership) {
        String clzName = lookupFromPropFile(countryCode + "." + membership.name());
        if (StringUtils.isBlank( clzName )) {
            clzName = lookupFromPropFile(countryCode);
        }
        return clzName;
    }
}

After restarting spring boot app, it works fine for the first couple of days, even with CA=CAOrderProcessor. But then one day, with countryCode=CA, it throws NoSuchBeanDefinitionException: No qualifying bean of type 'FedProvTaxCalculator' available: expected at least 1 bean which qualifies as autowire candidate. After restarting Java app, it works again for CA=CAOrderProcessor.

Why does spring framework behave this way? Thanks in advance!

The issue can be solved by

@Configuration public class TaxCalculatorConfiguration {
    @Bean("onTaxCalculator")
    public ITaxCalculator ontairioTaxCalculator() { ... }
}
public class CAOrderProcessor implements IOrderProcessor {
    @Autowire @Qualifier("onTaxCalculator")
    private ITaxCalculator onTaxCalculator;
}

Using AutowireCapableBeanFactory is fine. Why does it work initially, and then fails, and only fails on one ENV - openshift with min 2 pods? the other ENVs work fine always. Looks like spring relaxes autowire bean-type check initially, and later on under certain conditions, it checks the bean-type. Logical guess is that bean-definition returns interface type, which may be proxied, bean-wiring refers a concrete type, the proxied interface doesn't equal concrete type, raising this error. But in that case, it should always give error. If not, if I don't use the cache, or evict cache, I should be able to easily re-produce it in any ENVs, but it works fine on my local macos + oracle jdk 1.8. I even create a docker container based on production openshift docker image to run the app without cache, evict cache, force YGC and FGC, it works fine too.

Answer 1

I don't why behaves like that, probably because you use AutowireCapableBeanFactory directly and even worse then that in combination with @Cacheable .

You should reconsider your framework level code . I believe that you should never use AutowireCapableBeanFactory directly and especially in your case. It's simple and you can achieve the same result with less effort and using simple Map of country_code + membershi_type -> processor , for example:

@Configuration
public class ProcessorConfiguration {
    . . .
    @Bean("cAOrderProcessor ")
    public IOrderProcessor cAOrderProcessor() [
       return new CAOrderProcessor();
    }
    . . .
    @Bean
    public IOrderProcessorFactory processorFactory() {
       // create country_code + membershi_type -> processor map

       Map<ProcessorKey, IOrderProcessor> processorMap = new HashMap<>();
       // not sure about values in MembershipType, so I put SOME just for example
       // this map also can be a bean if you're gonna need that in other parts of app
       processorMap.put(new ProcessorKey("CA", MembershipType.SOME), cAOrderProcessor());

       // set it to factory
       return new OrderProcessorFactoryPropImpl(processorMap );
    }
    . . .
}

public class OrderProcessorFactoryPropImpl implements IOrderProcessorFactory {
    private final Map<ProcessorKey, IOrderProcessor> processorMap;

    public OrderProcessorFactoryPropImpl(Map<ProcessorKey, IOrderProcessor> processorMap) {
        this.processorMap = processorMap;
    }

    @Override
    // @Cacheable("orderProcessor") you dont need that because get it from map costs nothing
    // changed the name to "get" instead of "create"
    public IOrderProcessor getOrderProcessor(String countryCode, MembershipType membership) {
        // just get processor by key
        return processorMap.get(constructKey(countryCode, membership));
    }

    private ProcessorKey constructKey(String countryCode, MembershipType membership) {
        return new ProcessorKey(countryCode, membership);
    }
}

Also I noticed that you mix java and annotation-based bean config which is considered as a bad practice. Hope this's going to help.

---

Update 1 - Answering comment

Well, to figure out what's wrong person's gonna need to full copy of your app to debug/logs and reproduce it's usual use cases. It's probably not possible to say what's wrong just looking at examples you've provided (at least for me).

And I just pointed out that the way you are using AutowireCapableBeanFactory is not up to best practices and that's why you have problems in your runtime.

So you probably have 2 solutions:

1. Get rid of it, and use somewhat different approach (maybe similar to the one I've suggested previously). I believe that's only one good option. But this is up to you to decide.

2. Enable spring logs and hope that you will catch problem there. Probably need to enable debug logs like that in your log4j.xml (I suppose it's log4j but might be something else):

<category name="org.springframework.beans"> <priority value="debug" /> </category>