Message location/index in a queue (SQS or RabbitMQ)

Question

I'm interested in finding the location of a message in a FIFO queue. The goal is to be able to display to a user "You are 3rd in line", then "You are 2nd in line", then "Your request is being processed". I have access to AWS SQS and RabbitMQ. I know I can get the total number of messages in the queue, I just don't know where the message I care about is located. So far I have read docs on the queuing technologies, but haven't seen a solution, so no code has been written yet. Has anyone else been able to successfully find the index/location of a single message?

Answer 1

It isn't possible to get an item's "index".

What you can do though, is get an estimate of wait time (interestingly enough, this concept is very similar to how Disneyland measures wait time in their lines ):

1. Measure the following datapoints (and know when they change):
   • parallelism - How many items are processed in parallel?
   • processingTime - How long does it take an item to be processed?
   • queueLength - How many items are in the queue?
2. Calculate estimate:
   • waitTimeOfNextItem = queueLength * processingTime / parallelism

For example, if you have 100 items in the queue and each will take around 10 seconds to process, and you process 4 at once, then it'll take 100*10/4 or 250 seconds to process all the items. If a 101st item were to be put into the queue, you could estimate it would start processing after 250 seconds.

If your processingTime varies a lot, you can do calculations like rolling average or sampling, or provide a time range (5-10 minutes), depending on how accurate you want to be.

While you can also randomly send through "time card" items to refine this further (essentially a fake item that purely has the purpose of calculating how long it takes to get through the system), you can just send this info with every item instead.