Toggle routing in Rabbit MQ

Question

In RabbitMQ, is there a way to atomically toggle the binding of queues to exchange like the setup below. Q1 & Q2 are alternatingly bound.

X -> Q1
  -> Q2

By binding on an alternating basis, it creates the problem that messages will be either duplicated if Q1 & Q2 are bound at the same time during a toggle or lost if neither are. The requirement is to not lose or duplicate messages. I realise the window for lost messages will be very small, but I'd rather not deal with the logic issues that ensue.

Effectively what I want to achieve is a form of routing from exchange X to a queue (Q1 or Q2), but not make this a concern of the application that publishes the message, which publishes to an exchange X.

One way of achieving this is to have a process consuming message off a intermediate queue Q and implement the toggle routing in that process by giving Q1 and Q2 each an exchange of their own.

This strikes me as inefficient and introduces more moving parts to go wrong.

An alternate way would be to write a custom exchange that acts as a buffer/queue when no queues (X -> Q1 or X -> Q2) are bound, but my Erlang is non existent.

The easiest way would be to toggle the bindings between X -> Q1 and X -> Q2, but (afaik) there is no atomic toggling of bindings, messages could either get lost (unbind Q1, then bind Q2) or duplicated (Bind Q2, then unbind Q1).

The only workaround I can think of would be to temporarily make exchange X unavailable (ACL? rename?) prior to the binding toggle and hope the clients deal with the error conditions.

Is there an out-of-the-box way of achieving this cleanly? If not, what approach would you recommend?

Answer 1

Perhaps you want something like the consistent hashing exchange or the modulo hash exchange provided by the rabbitmq sharding plugin (also at the same link).

These plugins take care of partitioning the stream, which seems to be what you want to do actually.

Answer 2

Toggle bindings may be done with the help of Alternate Exchangs RabbitMQ extension:

1. Create extra exchange, say X-AE and bind Q2 to X-AE .
2. Setup X-AE as an Alternate Exchange on X exchange (to route all messages (that one that can't be handled) to Q2 queue).
3. Unbind Q1 from X (messages from will flow to Q2 via X-AE ).
4. Bind Q2 to X .
5. Remove alternate exchange X-AE policy from X exchange.
6. Unbind Q2 from X-AE and remove X-AE exchange.

Note, if you have some other Alternate Exchanges on your X exchange exact steps may vary, but the idea still the same.