RabbitMQ stops delivering MQTT messages to consumer after consumer restarts

Question

I have a RabbitMQ V3.6.10 instance on OpenShift running with the MQTT plugin enabled. And I have multiple Spring Boot applications with the Eclipse PAHO MQTT implementation consuming messages from a RabbitMQ queue. All consumers use MqttDefaultFilePersistence . The persistence data is written to a directory /tmp/mqtt on a persistent volume with a 100M quota. This is the code I wrote to connect to the RabbitMQ queues:

    MqttConnectOptions options = new MqttConnectOptions();
    options.setUserName(username);
    options.setPassword(password.toCharArray());
    options.setCleanSession(false);
    options.setAutomaticReconnect(reconnect);
    MqttClient mqttClient = new MqttClient(serverUri, "MyConsumerApp", new MqttDefaultFilePersistence("/tmp/mqtt"));
    mqttClient.connect(options);

Now I recognized that for some reasons if a cosumer app is restarted it does not consume any messages anymore. The messages in the queues are filling up as I can see in the RabbitMQ management console. There are no error messages in the RabbitMQ logs. The only solution that works in this case is deleting the subscription queue in the RabbitMQ management console and restarting the consumer application again. Maybe someone encountered similar issues? Does anyone have a hint for me what's going on here? Maybe there is an issue with the MQTT plugin RabbitMQ uses and the PAHO implementation holding message state. Looks like RabbitMQ does not know where to resume after consumer disconnected so it just stops delivering messages. Kinda pessimistic locking... :-)

Answer 1

You must be using persistent queues. If filling up of queue is the problem then you can use auto-delete queues. But this comes with a drawback that one attempt is made to deliver the message and if there is no consumer then queue is deleted and message is lost.

For you case you should look at Dead Letter Exchanges ( https://www.rabbitmq.com/dlx.html )

If consumer is cut off and reconnects after sometime, messages which were not processed can be served from Dead letter exchange. This way the order of message is also not lost.