Resetting memcache key data after a data change

Question

I've been reading around the web about memcache. I have found lots of examples which discuss the flow of setting and retrieving data with memcache such as.

REQUEST data key from memcache
IF key not found
THEN get data from DB and store result in memcache
ELSE get data from memcache

That's perfectly fine, straight forward and easy to understand. What I'm trying to work out though is the best approach of updating an already defined key after you make a change to the data set.

For example if I cache the following query:

SELECT * FROM user_info WHERE user = 26;

I would probably hash the SQL and use that as the key and stick the data in with it. But if I now go and update the user_info table for user 26, what is the best way of updating that cache data, I won't be able to know the hashed SQL string without going back to the query.

I did read that you run your SELECT queries again after a data change but that seems a bit awkward if you have a lot of queries referencing the same table.

Anyway, I'd like to hear workflow suggestions.

Answer 1

There are only two hard things in Computer Science: cache invalidation and naming things.

-- Phil Karlton

Knowing when to clear cache is extremely hard. There are many approaches to it, but I am not sure there is a silver bullet solution.

Some approaches to consider, depending on your exact use case:

1 - Key-based caching - change the cache key when the data changes - typically including a timestamp in the key: http://37signals.com/svn/posts/3113-how-key-based-cache-expiration-works

2 - Use keys that are programmatically easy to expire. eg in the case above the key could be

user_info_26

Then wrap any update to user_info in a single method that does

Update info for user 26
clear cache for key  user_info_26

And then treat that as an atomic operation in your code.

3 - Have some sort of cache sweeper asynchronous job. eg Periodically check the last updated timestamp of user 26, and if it doesn't match the cache data clear the cache.

I am sure there are many other strategies, but the correct one will depend on the exact nature of your application.