Web application, clustered tomcats and mongoDB - how to achieve a persistance queue

Question

We are building a small web application that will run on a cluster of 3 tomcats with a load balancer, the web app is based on the spring framework with spring data module to for the mongodb.

Our goal is to have a queue with persistancy (will retain data on cluster failure \\ server shut down etc).

This queue must be emptied if its size > 0.

We would like that some process will try to empty this queue if the queue size>0

here are some things to consider:

1) in order to remove an element from the queue , some amount of logic is needed. 2) it is very likely that after some amount of logic is done for a specific queue-element, the resultion will be NOT to remove the element from the queue. (to be later on retried) 3) we are using a cluster of tomcats, we must retain a concurrency mechanism for things not to go wrong. (no locking is supported by mongodb)

any ideas whats the easiest way to achieve this?

thanks!

Answer 1

Use ActiveMQ between Tomcat and Mongo. Have one queue for requests. Get the value from the queue and process it. If retry is needed, use AMQs retry mechanism and add the object to the same queue.