How to have multiple round robin RabbmitMQ consumers without concurrency

Question

I am currently working on putting together a NodeJS system that is responsible for receiving a large amount of events and the order of those events being processed is highly critical. It is also important that the application scales and can handle a Rabbit consumer falling over, therefore I have multiple consumers reading off a queue which is bound to a direct exchange with 'noAck' set to false and each queue having a prefetch count of 1.

This is ensuring that my messages are being processed in order however both consumers are processing events concurrently, where my desired outcome is:

Consumer A          Consumer B
----------          -----------
process event 1
...
acknowledge
                    process event 2
                    ... 
                    acknowledge
process event 3
...
..and so on.

I realise that this reduces the efficiency of my nodes but the guarantee that events are fully processed in order is much more critical to me.

Any help would be greatly appreciated.

Answer 1

This is ensuring that my messages are being processed

No, whenever you have concurrent consumers, settings basic_qos=1 won't prevent other consumers from processing messages out of order.

If you need ordered processing you must have 1 consumer per queue. If you want parallelism, the best you could do is to partition the message stream into several queues, using something like the Consistent Hash Exchange . Of course messages will lose order at partitioning time, but then one consumer per queue will assure ordered processing on each queue.