How to Redis cache Large C# object without serializing?

Question

In my application we were using C# dictionary to in-memory cache a large Business object.

I am in the middle of converting the huge multiple round referenced C# object(~300KB when JSON Serialized) to DTOs for the purpose of JSON Serializing/Deserializing for Redis caching.

It seem like some Types in my business layer cannot be easily initialized due to parameterized constructors.

Is there a way I can Redis cache without serialization? Can this way give back the same object I cached.

Answer 1

Is there a way I can Redis cache without serialization?

The short answer: No .

The slightly longer answer: No, because Redis is based on strings. In those strings, you can store all kinds of things, but in the end it all boils down to strings. So you can't get out of serializing when you want to store something in Redis.

Redis is not a plain key-value store, it is actually a data structures server, supporting different kinds of values. What this means is that, while in traditional key-value stores you associated string keys to string values, in Redis the value is not limited to a simple string, but can also hold more complex data structures. The following is the list of all the data structures supported by Redis, which will be covered separately in this tutorial:

• Binary-safe strings.
• Lists: collections of string elements sorted according to the order of insertion. They are basically linked lists.
• Sets: collections of unique, unsorted string elements.
• Sorted sets, similar to Sets but where every string element is associated to a floating number value, called score. The elements are always taken sorted by their score, so unlike Sets it is possible to retrieve a range of elements (for example you may ask: give me the top 10, or the bottom 10).
• Hashes, which are maps composed of fields associated with values. Both the field and the value are strings. This is very similar to Ruby or Python hashes.
• Bit arrays (or simply bitmaps): it is possible, using special commands, to handle String values like an array of bits: you can set and clear individual bits, count all the bits set to 1, find the first set or unset bit, and so forth.
• HyperLogLogs: this is a probabilistic data structure which is used in order to estimate the cardinality of a set. Don't be scared, it is simpler than it seems... See later in the HyperLogLog section of this tutorial.

More information: An introduction to Redis data types and abstractions

Answer 2

Okay. I know it is a long time after I asked this question. But we solved this. We used BinaryFormatter Serializer which essentially converts the .NET object into a memory map and serializes it as a binary array. This memory map can be used to recreate the original object without public default constructors. This BinaryFormatter serialization is supported by .NET from early versions.

The downside to this approach is that we have to manually go to every class in the object hierarchy and give Serializable Attribute decoration to the class. Another downside is that only the properties that hold a non-null value get serialized. This means, If none of your unit tests initialize some or any property of the object being serialized, there is a great chance we may hit a runtime Serialization Exception, which can be easily solved by decoration the type Serializable

We may also be able to use, other serialization formats like MsgPack etc.

public class TheTypeToBeCached
{
   public OneSubType Property1 {get;set;}
   public SecondSubType Property2 {get;set;}
}

public class OneSubType 
{
    public ThirdSubType Property3 {get;set;}
    public ForthSubType Property4 {get;set;}
    public AnotherSubType Property5 {get;set;}
}
public class SecondSubType
{
    public ForthSubType Property6 {get;set;}
    public AnotherSubType Property7 {get;set;}
}
public class ThridSubType
{
    public SecondSubType Property8{get;set;}
}
public class ForthSubType
{
    public SecondSubType Property9{get;set;}
}
public class AnotherSubType
{
    public OneSubType Property10{get;set;}
}

The above hierarchy can be Serialized and deserialzed using BinaryFormatter Serializer even if it has multiple round relationships.

[Serializable]
public class TheTypeToBeCached
{
   public OneSubType Property1 {get;set;}
   public SecondSubType Property2 {get;set;}
}

[Serializable]
public class OneSubType 
{
    public ThirdSubType Property3 {get;set;}
    public ForthSubType Property4 {get;set;}
    public AnotherSubType Property5 {get;set;}
}

[Serializable]
public class SecondSubType
{
    public ForthSubType Property6 {get;set;}
    public AnotherSubType Property7 {get;set;}
}

[Serializable]
public class ThridSubType
{
    public SecondSubType Property8{get;set;}
}

[Serializable]
public class ForthSubType
{
    public SecondSubType Property9{get;set;}
}

[Serializable]
public class AnotherSubType
{
    public OneSubType Property10{get;set;}
}