Serializing List<Dictionary<string,object>> with Protobuf-Net

Question

I've got a list of dictionaries that I want to serialise but I get the following error:

System.NotSupportedExceptionNested or jagged lists and arrays are not supported

Is it even possible to serialise this? I've tried pushing this to a different class and have a List<Dto> where the Dto class is an IEnumerable of the Dictionary , with a DynamicType for the object in the dictionary, but it still doesn't work. Am I going about this completely the wrong way or am I missing something subtle?

Example code:

var zone = DateTimeZoneProviders.Tzdb["Europe/London"];
        _testObj = new Person
        {
            Name = "Homer",
            Age = 38,
            Timestamp = new ZonedDateTime(Instant.FromUtc(2013, 6, 12, 17, 53, 23), zone),
            Target = new DataCollection(new Dictionary<string, Type>()
            {
                {"Date", typeof(DateTime)}

            }, new List<Dictionary<string, object>>
            {
                new Dictionary<string, object>
                {
                    {"Date", new DateTime(2015, 1, 1)}
                }
            }
            , new List<string> { "Date" })

And the DataCollection :

[Serializable, ProtoContract(IgnoreListHandling = true), DataContract, JsonObject]
public class DataCollection : IEnumerable<IDictionary<string, object>>
{
    public DataCollection(Dictionary<string, Type> fields, List<Dictionary<string, object>> data, List<string> keyFields)
    {
        Fields = fields;
        KeyFields = keyFields;
        Data = data;
    }

    public DataCollection()
    {
        Fields = new Dictionary<string, Type>();
        Data = new List<Dictionary<string, object>>();
        KeyFields = new List<string>();
    }

    [ProtoMember(1), DataMember]
    public Dictionary<string, Type> Fields { get; set; }

    [ProtoMember(2), DataMember]
    public List<string> KeyFields { get; set; }

    [ProtoMember(3), DataMember]
    public List<Dictionary<string, object>> Data { get; set; }

    public void Add(Dictionary<string, object> value)
    {
        Data.Add(value);
    }

    public IEnumerator<IDictionary<string, object>> GetEnumerator()
    {
        return Data.GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

I can serialise just fine if I don't have ProtoMember(3) included.

Answer 1

So the issue came down to Protobuf-net, rightly so, not liking the object Type in the Dictionary. I won't go to the specifics as to why I need that there, suffice to say you need to use a combination of 2 surrogate classes, and the object Type will need to be serialised as byte[] with the BinaryFormatter and then sent to Protobuf-net, as the Protobuf spec won't acknowledge an object Type.

I am copying the inspiration for the solution from this website: http://rushfrisby.com/serializing-non-serializable-types-with-protobuf/

according to the rules of posting solutions and not links to external sites alone:

---

Take this class for example:

[DataContract]
public class MyNameValueInfo
{
    [DataMember(Order = 1)]
    public string Name { get; set; }

    [DataMember(Order = 2)]
    public object Value { get; set; }
}

MyNameValueInfo can't be serialized because it doesn't know how to serialize the Value property (typeof object). It will throw an exception: “No Serializer defined for type: System.Object”

To get around this we need to provide a surrogate for MyNameValueInfo that protobuf-net can serialize. First register the surrogate type (only needs to be done once):

RuntimeTypeModel.Default.Add(typeof(MyNameValueInfo), false).SetSurrogate(typeof(MyNameValueInfoSurrogate));

Then implement MyNameValueInfoSurrogate so that it can be transformed from/to MyNameValueInfo and is serializable by protobuf-net:

[DataContract]
public class MyNameValueInfoSurrogate 
{
        //string is serializable so we'll just copy this property back and forth
    [DataMember(Order = 1)]
    public string Name { get; set; } 

        //byte[] is serializable so we'll need to convert object to byte[] and back again
    [DataMember(Order = 2)]
    public byte[] Value { get; set; }

    public static implicit operator MyNameValueInfo(MyNameValueInfoSuggorage suggorage)
    {
        return suggorage == null ? null : new MyNameValueInfo
        {
            Name = suggorage.Name,
            Value = Deserialize(suggorage.Value)
        };
    }

    public static implicit operator MyNameValueInfoSuggorage(MyNameValueInfo source)
    {
        return source == null ? null : new MyNameValueInfoSuggorage
        {
            Name = source.Name,
            Value = Serialize(source.Value)
        };
    }

    private static byte[] Serialize(object o)
    {
        if (o == null)
            return null;

        using (var ms = new MemoryStream())
        {
            var formatter = new BinaryFormatter();
            formatter.Serialize(ms, o);
            return ms.ToArray();
        }
    }

    private static object Deserialize(byte[] b)
    {
        if (b == null)
            return null;

        using (var ms = new MemoryStream(b))
        {
            var formatter = new BinaryFormatter();
            return formatter.Deserialize(ms);
        }
    }
}