C#: How to implement a smart cache

Question

I have some places where implementing some sort of cache might be useful. For example in cases of doing resource lookups based on custom strings, finding names of properties using reflection, or to have only one PropertyChangedEventArgs per property name.

A simple example of the last one:

public static class Cache
{
    private static Dictionary<string, PropertyChangedEventArgs> cache;
    static Cache()
    {
        cache = new Dictionary<string, PropertyChangedEventArgs>();
    }
    public static PropertyChangedEventArgs GetPropertyChangedEventArgs(
        string propertyName)
    {
        if (cache.ContainsKey(propertyName))
            return cache[propertyName];

        return cache[propertyName] = new PropertyChangedEventArgs(propertyName);
    }
}

But, will this work well? For example if we had a whole load of different propertyNames, that would mean we would end up with a huge cache sitting there never being garbage collected or anything. I'm imagining if what is cached are larger values and if the application is a long-running one, this might end up as kind of a problem... or what do you think? How should a good cache be implemented? Is this one good enough for most purposes? Any examples of some nice cache implementations that are not too hard to understand or way too complex to implement?

Answer 1

This is a large problem, you need to determine the domain of the problem and apply the correct techniques. For instance, how would you describe the expiration of the objects? Do they become stale over a fixed interval of time? Do they become stale from an external event? How frequently does this happen? Additionally, how many objects do you have? Finally, how much does it cost to generate the object?

The simplest strategy would be to do straight memoization, as you have above. This assumes that objects never expire, and that there are not so many as to run your memory dry and that you think the cost to create these objects warrants the use of a cache to begin with.

The next layer might be to limit the number of objects, and use an implicit expiration policy, such as LRU (least recently used). To do this you'd typically use a doubly linked list in addition to your dictionary, and every time an objects is accessed it is moved to the front of the list. Then, if you need to add a new object, but it is over your limit of total objects, you'd remove from the back of the list.

Next, you might need to enforce explicit expiration, either based on time, or some external stimulus. This would require you to have some sort of expiration event that could be called.

As you can see there is alot of design in caching, so you need to understand your domain and engineer appropriately. You did not provide enough detail for me to discuss specifics, I felt.

PS Please consider using Generics when defining your class so that many types of objects can be stored, thus allowing your caching code to be reused.

Answer 2

You could wrap each of your cached items in a WeakReference . This would allow the GC to reclaim items if-and-when required, however it doesn't give you any granular control of when items will disappear from the cache, or allow you to implement explicit expiration policies etc.

(Ha! I just noticed that the example given on the MSDN page is a simple caching class.)

Answer 3

Looks like .NET 4.0 now supports System.Runtime.Caching for caching many types of things. You should look into that first, instead of re-inventing the wheel. More details:

http://msdn.microsoft.com/en-us/library/system.runtime.caching%28VS.100%29.aspx

Answer 4

This is a nice debate to have, but depending your application, here's some tips:

You should define the max size of the cache, what to do with old items if your cache is full, have a scavenging strategy, determine a time to live of the object in the cache, does your cache can/must be persisted somewhere else that memory, in case of application abnormal termination, ...

Answer 5

This is a common problem that has many solutions depending on your application need. It is so common that Microsoft released a whole library to address it. You should check out Microsoft Velocity before rolling up your own cache. http://msdn.microsoft.com/en-us/data/cc655792.aspx Hope this help.

Answer 6

You could use a WeakReference but if your object is not that large than don't because the WeakReference would be taking more memory than the object itself which is not a good technique. Also, if the object is a short-time usage where it will never make it to generation 1 from generation 0 on the GC, there is not much need for the WeakReference but IDisposable interface on the object would have with the release on SuppressFinalize .

If you want to control the lifetime you need a timer to update the datetime/ timespan again the desiredExpirationTime on the object in your cache.

The important thing is if the object is large then opt for the WeakReference else use the strong reference. Also, you can set the capacity on the Dictionary and create a queue for requesting additional objects in your temp bin serializing the object and loading it when there is room in the Dictionary, then clear it from the temp directory.