Does the RabbitMQ shovel plugin preserve message ordering?

Question

I use the RabbitMQ shovel plugin (dynamic shovel, see below) to provide unidirectional messaging between two RabbitMQ brokers over an unreliable WAN link. I see regular connection losses in the RabbitMQ server log.

The relevant portions of the AMQP setup are identical for both brokers: one exchange (fanout, durable) and one queue (durable). The consuming application requires that messages are received in the same order they are produced at the sending side.

The observed behaviour seems to indicate that this is not the case, perhaps due to retransmissions, etc. Does the RabbitMQ shovel pluging preserve message ordering without message loss? What are the required configuration options?

Answer 1

For ensuring message ordering the following parameters should be configured:

 "prefetch-count" : 1
 "ack-mode"       : "on-confirm"

prefetch-count The maximum number of unacknowledged messages copied over a shovel at any one time. Default is 1000.

ack-mode Determines how the shovel should acknowledge messages. If set to on-confirm (the default), messages are acknowledged to the source broker after they have been confirmed by the destination. This handles network errors and broker failures without losing messages, and is the slowest option. If set to on-publish, messages are acknowledged to the source broker after they have been published at the destination. This handles network errors without losing messages, but may lose messages in the event of broker failures. If set to no-ack, message acknowledgements are not used. This is the fastest option, but may lose messages in the event of network or broker failures.

For more information related to RabbitMQ's shovel mechanism please refer to the official documentation: https://www.rabbitmq.com/shovel-dynamic.html