一旦消息数量超过并发消费者数量，消费者就不会被分配给消息

Question

I want to understand how the consumers work for concurrent consumer once a message is consumed. Is the consumer bean discarded and new one created or is the consumer bean reused? I can see after rabbit mq server is set up, when I send a message, a consumer bean is instantiated.

My issue is that once the number of messages equals the number of concurrent consumers, the consumer in the message listener is null for the next message. I can see the Ack/Nack for previous messages in my logs and the prefetch is set to 1.

XML configuration :

<rabbit:admin connection-factory="cachingConnectionFactory" />

<rabbit:template id="template" connection-factory="cachingConnectionFactory"
    retry-template="retryTemplate" />

<!-- CachingConnectionFactory -->
<bean id="cachingConnectionFactory"
    class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
    <constructor-arg ref="clientConnectionFactorySSL" />
    <property name="username" value="${QM_USERNAME}" />
    <property name="password" value="${QM_PASSWORD}" />
    <property name="virtualHost" value="${QM_VIRTUALHOST}" />
    <property name="host" value="${QM_HOST}" />
    <property name="port" value="${QM_PORT}" />
</bean>

<!-- SimpleRabbitListenerContainerFactory -->
        <bean id="rabbitListenerContainerFactory"
            class="org.springframework.amqp.rabbit.config.SimpleRabbitListenerContainerFactory">
            <property name="connectionFactory">
                <ref bean="cachingConnectionFactory" />
            </property>
            <property name="concurrentConsumers" value="5" />
            <property name="maxConcurrentConsumers" value="8" />
             <property name="startConsumerMinInterval" value="3000" />
        </bean>

<!-- Message listener container -->
        <bean id="springListenerContainerRABC"          class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
            <property name="connectionFactory" ref="cachingConnectionFactory" />
            <property name="queues" ref="rabbitDestinationRABC" />
            <property name="messageListener" ref="rabbitMessageListenerRABC" />
            **<property name="concurrentConsumers" value="4" />**
            <property name="maxConcurrentConsumers" value="5" />
        </bean>

<!-- Message Listener -->
    <bean id="rabbitMessageListenerRABC" class="com.queue.rabbitmessaging.RabbitMQListener">
        **<property name="consumerType" value="XYZConsumer" />**
        <property name="responseSender" ref="XYZResponseSender" />
        <property name="taskExecutor" ref="taskExecutor" />
        <property name="encoding" value="UTF-16BE" />
    </bean>

<!-- Consumers -->
<bean id="XYZConsumer" class="com.msg.consumer.XYZConsumer"
    scope="prototype" />

Here is what I see in the logs :

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - **Received message for consumer: XYZConsumer**

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Message :: test1

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Received message for consumer: XYZConsumer

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Message :: test2

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Received message for consumer: XYZConsumer

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Message :: test3

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Received message for consumer: XYZConsumer

00:17:49.725 [springListenerContainerRABC-1] INFO RabbitMQListener - Message :: test4

[org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] INFO RabbitMQListener - **Received message for consumer: null**

[org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] INFO RabbitMQListener - Message :: test5

Consumer Code is as follows:

@EnableRabbit
@Service 
public class RabbitMQListener implements MessageListener {
    @Autowired
    private ApplicationContext applicationContext;

    protected static Logger logger = LoggerFactory.getLogger(RabbitMQListener.class);

    private TaskExecutor taskExecutor;
    private RabbitSender responseSender;
    private String consumerType;
    private String encoding;

     public TaskExecutor getTaskExecutor() {
        return taskExecutor;
    }

    public void setTaskExecutor(TaskExecutor taskExecutor) {
        this.taskExecutor = taskExecutor;
    }

    public void setResponseSender(RabbitSender responseSender) {
        this.responseSender = responseSender;
    }

    public void setConsumerType(String consumerType) {
        this.consumerType = consumerType;
    }

    public void setEncoding(String encoding) {
        this.encoding = encoding;
    }

    @Override
    @RabbitListener(containerFactory = Constants.CONTAINER_FACTORY, queues = "#{'${QM_QUEUES}'.split(',')}")
    public void onMessage(final Message arg0) {

        if (arg0 != null) {
            try {
                String message = new String(arg0.getBody());
                logger.info("Message Properties class: " + arg0.getClass());
                logger.info("Message Properties: " + arg0.getMessageProperties());
                logger.info("Received message for consumer: " + this.consumerType);
                logger.info("Message :: " + message);
                Object bean = applicationContext.getBean(this.consumerType);
                if (bean instanceof Consumer) {
                    logger.debug("Processing message for consumer: " + this.consumerType);
                    Consumer consumer = (Consumer)bean;
                    consumer.setMessage(message);
                    consumer.setResponseSender(responseSender);
                    logger.debug("Executing consumer: " + this.consumerType);
                    taskExecutor.execute(consumer);
                } else {
                    logger.info("Could not process message due to invalid consumer.");
                }

            } 
            catch (AmqpException e1) {
                logger.error("Spring AMQP error processing message.", e1.getMessage());
            }
            catch (Exception e2) {
                logger.error("General error processing message.", e2.getMessage());
            }
        }
        else {
            logger.debug("message received is NULL");
        }

    }


}

Answer 1

The described behavior is consistent with not acking messages after they are received. RabbitMQ can be configured to automatically acknowledge a message upon delivery. If this does not happen, or is not configured, RabbitMQ will assume that the consumer is busy and will not deliver additional messages to it. You have to send an ack following the successful processing of a message to be able to receive the next message in the queue.