Spring JMS listener for AWS SQS queue configuration with concurrency

Question

I am trying to figure out how to configure jms listener to listen AWS queue and process messages in many threads (~100 simultaneously).

Below is my configuration.

@Configuration
@EnableJms
public class JmsConfig {

    @Bean
    public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(AmazonSQS amazonSQS) {
        ProviderConfiguration providerConfiguration = new ProviderConfiguration().withNumberOfMessagesToPrefetch(0);
        SQSConnectionFactory connectionFactory = new SQSConnectionFactory(providerConfiguration, amazonSQS);
        DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
        factory.setConnectionFactory(connectionFactory);
        factory.setDestinationResolver(new DynamicDestinationResolver());
        factory.setConcurrency("30-100");
        factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
        factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_CONNECTION);
        factory.setErrorHandler(t -> {
        });
        return factory;
    }

}

With this confuguration I keep receiving the following error:

SQSMessageConsumer - Can't terminate executor service ConsumerPrefetch after 30 seconds, some running threads will be shutdown immediately

Also, publishing messages into AmazonSQS instance takes 20 seconds.

I tried different combinations of NumberOfMessagesToPrefetch and CacheLevel but nothing works properly.

Eg CacheLevel = CACHE_CONSUMER works with no errors but processes 1 message at a time.

Please help me to configure this. Thanks!

Libraries:

• aws-java-sdk:1.11.41

• spring-jms:5.1.7

• amazon-sqs-java-messaging-lib:1.0.6

Answer 1

@ Boris I was able to resolve this issue the following way. I do not set Cache Level. The provider configuration is still disabling prefetching

ProviderConfiguration providerConfiguration = new ProviderConfiguration().withNumberOfMessagesToPrefetch(0);

Additionally, I use DefaultJmsListenerContainerFactory and then set the following values, in addition to other values not shown:

---

DefaultJmsListenerContainerFactory

factory.setConcurrency("5-10"); //This will depend on your use case factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE); //Used to in-flight erroneous messages factory.setTaskExecutor(threadPoolTaskExecutor()); //Uses threadpooltask executor, see below factory.setMaxMessagesPerTask(10); //Not required, but to reduce thread-switching. This will depend on your use case

---

Please read ThreadPool Executor before using ThreadPool Executor, written almost 12 years ago but still valid. I do not set concurrency on the @JmsListener level because it causes additional complexity JmsListener Concurrency