distributed Cache Architecture

Question

In my application, I am getting feeds(in the form of XML). I need to push these feeds to memcached. The feed expected to last for 24 hours. The reason being, there are some matching process need to happen on this cache. My problem is if the Cache server crashes, all my data is lost. I can't retrieve it. I am thinking to have a messaging queue, so that the feed comes first to messaging queue and then flows to Memcached Server. So that even if the cache server crashes, the data is still with messaging queue.

Is this is a better solution and there can be a better solution to it.

Regards, Debasish

Answer 1

Message queue guarantee delivery of messages in FIFO order, in general. The queue will discard the message (and should) once the client responds with a success status. So, it cannot be used as persistent store.

You need a persistent store like database to back up the messages and load them into the cache during failure recovery phase.