XML Serializing dapper results

Question

I store SQL results into a dynamic List, of which has an underlying DapperRow type. I am trying to serialize/unserialize this List of which XMLserializer complains:

There was an error generating the XML document. ---> System.InvalidOperationException: To be XML serializable, types which inherit from IEnumerable must have an implementation of Add(System.Object) at all levels of their inheritance hierarchy. Dapper.SqlMapper+DapperRow does not implement Add(System.Object).

Is there a way around this (besides the obvious casting the results to my own concrete object), or is it possible to somehow make DapperRow objects conform to System.Xml.XMLserializer constraints?

It states my result array is System.Collections.Generic.List<dynamic> {System.Collections.Generic.List<object>} Opening the array it says each object is of type object {Dapper.SqlMapper.DapperRow}

I think because DapperRows are now basically IDictionary<string, object> that XML is having issues (I cannot use anything but System.Xml.XmlSerializer so don't suggest an alternative).

I just want to be able to turn a List<dynamic> that I get from Dapper and serialize and deserialize correctly using System.Xml.XmlSerializer

Answer 1

I was able to get something at least serializable by using a serializable dictionary : http://weblogs.asp.net/pwelter34/444961

            var results = conn.Query<dynamic>(sql, param);
            var resultSet = new List<SerializableDictionary<string, object>>();
            foreach (IDictionary<string, object> row in results)
            {
                var dict = new SerializableDictionary<string, object>();
                foreach (var pair in row)
                {
                    dict.Add(pair.Key, pair.Value);
                }
                resultSet.Add(dict);
            }

Its ugly, so I hope more elegant solutions come up

Answer 2

Firstly, decorate the DapperResultSet with a [Serializable] attribute. Also create a constructor and in that, assign Rows with an empty List<object> . Use this modified code: (it also contains a modified implemented method)

[Serializable]
public class DapperResultSet : IEnumerable<object>

{ public List Rows { get; set; }

public void Add(dynamic o)
{
    Rows.Add(o);
}

public DapperResultSet()
{
    Rows = new List<object>();
}

public IEnumerator<object> GetEnumerator()
{
    return Rows.GetEnumerator();
}

System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
    return GetEnumerator();
}

}

Next in your event handler (or where you would like to do the serialization):

var results = conn.Query<dynamic>(sql, param);
var r = new DapperResultSet();
foreach (var row in results)
{
    r.Add(row);
}
//Here is the serialization part:
XmlSerializer xs = new XmlSerializer(typeof(DapperResultSet));
xs.Serialize(new FileStream("Serialized.xml", FileMode.Create), r); //Change path if necessary

For Deserialization,

XmlSerializer xs = new XmlSerializer(typeof(DapperResultSet));
DapperResultSet d_DRS = xs.Deserialize(new FileStream("Serialized.xml", FileMode.Open)); //Deserialized

Answer 3

is it possible to somehow make DapperRow objects conform to System.Xml.XMLserializer constraints?

I don't think that this is possible. The DapperRow class is private and it does not have a parameterless constructor.

However, you might be able to use other means to fix your problem.

I suggest that you put your complex serialization logic behind an extension method . This way your original code will stay clean.

I also suggest the following model for storing the data (although you can choose not to use it and use your own logic behind the extension method):

public class Cell
{
    public string Name { get; set; }
    public object Value { get; set; }

    public Cell(){}

    public Cell(KeyValuePair<string, object> kvp)
    {
        Name = kvp.Key;
        Value = kvp.Value;
    }
}

public class Row : List<Cell>
{
    public Row(){}

    public Row(IEnumerable<Cell> cells)
        : base(cells){}        
}

public class Rows : List<Row>
{
    public Rows(){}

    public Rows(IEnumerable<Row> rows )
        :base(rows){}
}

And then the extension method should look something like this:

public static class Extensions
{
    public static void Serialize(this IEnumerable<dynamic> enumerable, Stream stream)
    {
        var rows =
            new Rows(
                enumerable
                    .Cast<IEnumerable<KeyValuePair<string, object>>>()
                    .Select(row =>
                        new Row(row.Select(cell => new Cell(cell)))));

        XmlSerializer serializer = new XmlSerializer(typeof(Rows));

        serializer.Serialize(stream, rows);
    }
}

Then you would be able to use this code:

var result = connection.Query("SELECT * From Customers");

var memory_stream = new MemoryStream();

result.Serialize(memory_stream);

See how this code is very small because all the complex logic is moved to the extension method.

The model that I suggested allows also for deserialization, just make sure that you use the correct type (eg Rows ) like this:

XmlSerializer serializer = new XmlSerializer(typeof(Rows));

Rows rows = (Rows)serializer.Deserialize(stream);

You can also have an extension method that just converts the resultset of Dapper to the Rows type and handle the serialization of Rows your self. Such extension method should look something like this:

public static Rows ToRows(this IEnumerable<dynamic> enumerable)
{
    return
        new Rows(
            enumerable
                .Cast<IEnumerable<KeyValuePair<string, object>>>()
                .Select(row =>
                    new Row(row.Select(cell => new Cell(cell)))));
}

And then use it like this:

var rows = connection.Query("SELECT * From Customers").ToRows();

XmlSerializer serializer = new XmlSerializer(typeof(Rows));

serializer.Serialize(stream, rows);

Answer 4

Challenging request, because Dapper is not designed to be serializable. But let see what can be done.

The first decision is easy - we need to implement IXmlSerializable . The question is how.

Serialization is not a big deal, since we have the field names and values. So we could use similar approach to the SerializableDictionary<TKey, TValue> you mentioned. However, it heavily relies on typeof(TKey) and typeof(TValue) '. We have no problem with key (it's a string), but the type of the value is object . As I mentioned, it's not a problem to write an object value as XML. The problem is deserialization. At that point, all we have is a string and no any clue what that string is. Which means we need to store some metadata in order to be able to deserialize correctly. Of course there are many ways to do that, but I decided to store field names and types separately at the beginning, and then items with values only.

Putting it all together, here is what I ended up:

using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq.Expressions;
using System.Reflection;
using System.Xml;
using System.Xml.Schema;
using System.Xml.Serialization;

namespace Samples
{
    public class DapperResultSet : IXmlSerializable
    {
        static readonly Type TableType;
        static readonly Type RowType;
        static readonly Func<object, string[]> GetFieldNames;
        static readonly Func<object, object[]> GetFieldValues;
        static readonly Func<string[], object> CreateTable;
        static readonly Func<object, object[], object> CreateRow;
        static DapperResultSet()
        {
            TableType = typeof(Dapper.SqlMapper).GetNestedType("DapperTable", BindingFlags.NonPublic);
            RowType = typeof(Dapper.SqlMapper).GetNestedType("DapperRow", BindingFlags.NonPublic);
            // string[] GetFieldNames(object row)
            {
                var row = Expression.Parameter(typeof(object), "row");
                var expr = Expression.Lambda<Func<object, string[]>>(
                    Expression.Field(Expression.Field(Expression.Convert(row, RowType), "table"), "fieldNames"),
                    row);
                GetFieldNames = expr.Compile();
            }
            // object[] GetFieldValues(object row)
            {
                var row = Expression.Parameter(typeof(object), "row");
                var expr = Expression.Lambda<Func<object, object[]>>(
                    Expression.Field(Expression.Convert(row, RowType), "values"),
                    row);
                GetFieldValues = expr.Compile();
            }
            // object CreateTable(string[] fieldNames)
            {
                var fieldNames = Expression.Parameter(typeof(string[]), "fieldNames");
                var expr = Expression.Lambda<Func<string[], object>>(
                    Expression.New(TableType.GetConstructor(new[] { typeof(string[]) }), fieldNames),
                    fieldNames);
                CreateTable = expr.Compile();
            }
            // object CreateRow(object table, object[] values)
            {
                var table = Expression.Parameter(typeof(object), "table");
                var values = Expression.Parameter(typeof(object[]), "values");
                var expr = Expression.Lambda<Func<object, object[], object>>(
                    Expression.New(RowType.GetConstructor(new[] { TableType, typeof(object[]) }),
                        Expression.Convert(table, TableType), values),
                    table, values);
                CreateRow = expr.Compile();
            }
        }
        static readonly dynamic[] emptyItems = new dynamic[0];
        public IReadOnlyList<dynamic> Items { get; private set; }
        public DapperResultSet()
        {
            Items = emptyItems;
        }
        public DapperResultSet(IEnumerable<dynamic> source)
        {
            if (source == null) throw new ArgumentNullException("source");
            Items = source as IReadOnlyList<dynamic> ?? new List<dynamic>(source);
        }
        XmlSchema IXmlSerializable.GetSchema() { return null; }
        void IXmlSerializable.WriteXml(XmlWriter writer)
        {
            if (Items.Count == 0) return;
            // Determine field names and types
            var fieldNames = GetFieldNames((object)Items[0]);
            var fieldTypes = new TypeCode[fieldNames.Length];
            for (int count = 0, i = 0; i < Items.Count; i++)
            {
                var values = GetFieldValues((object)Items[i]);
                for (int c = 0; c < fieldTypes.Length; c++)
                {
                    if (fieldTypes[i] == TypeCode.Empty && values[c] != null)
                    {
                        fieldTypes[i] = Type.GetTypeCode(values[c].GetType());
                        if (++count >= fieldTypes.Length) break;
                    }
                }
            }
            // Write fields
            writer.WriteStartElement("Fields");
            writer.WriteAttributeString("Count", XmlConvert.ToString(fieldNames.Length));
            for (int i = 0; i < fieldNames.Length; i++)
            {
                writer.WriteStartElement("Field");
                writer.WriteAttributeString("Name", fieldNames[i]);
                writer.WriteAttributeString("Type", XmlConvert.ToString((int)fieldTypes[i]));
                writer.WriteEndElement();
            }
            writer.WriteEndElement();
            // Write items
            writer.WriteStartElement("Items");
            writer.WriteAttributeString("Count", XmlConvert.ToString(Items.Count));
            foreach (IDictionary<string, object> item in Items)
            {
                writer.WriteStartElement("Item");
                foreach (var entry in item)
                {
                    writer.WriteStartAttribute(entry.Key);
                    writer.WriteValue(entry.Value);
                    writer.WriteEndAttribute();
                }
                writer.WriteEndElement();
            }
            writer.WriteEndElement();
        }
        void IXmlSerializable.ReadXml(XmlReader reader)
        {
            reader.MoveToContent();
            bool isEmptyElement = reader.IsEmptyElement;
            reader.ReadStartElement(); // Container
            if (isEmptyElement) return;
            // Read fields
            int fieldCount = XmlConvert.ToInt32(reader.GetAttribute("Count"));
            reader.ReadStartElement("Fields");
            var fieldNames = new string[fieldCount];
            var fieldTypes = new TypeCode[fieldCount];
            var fieldIndexByName = new Dictionary<string, int>(fieldCount);
            for (int c = 0; c < fieldCount; c++)
            {
                fieldNames[c] = reader.GetAttribute("Name");
                fieldTypes[c] = (TypeCode)XmlConvert.ToInt32(reader.GetAttribute("Type"));
                fieldIndexByName.Add(fieldNames[c], c);
                reader.ReadStartElement("Field");
            }
            reader.ReadEndElement();
            // Read items
            int itemCount = XmlConvert.ToInt32(reader.GetAttribute("Count"));
            reader.ReadStartElement("Items");
            var items = new List<dynamic>(itemCount);
            var table = CreateTable(fieldNames);
            for (int i = 0; i < itemCount; i++)
            {
                var values = new object[fieldCount];
                if (reader.MoveToFirstAttribute())
                {
                    do
                    {
                        var fieldName = reader.Name;
                        var fieldIndex = fieldIndexByName[fieldName];
                        values[fieldIndex] = Convert.ChangeType(reader.Value, fieldTypes[fieldIndex], CultureInfo.InvariantCulture);
                    }
                    while (reader.MoveToNextAttribute());
                }
                reader.ReadStartElement("Item");
                var item = CreateRow(table, values);
                items.Add(item);
            }
            reader.ReadEndElement(); // Items
            reader.ReadEndElement(); // Container
            Items = items;
        }
    }
}

Some things would have been easier if we modify the Dapper source code, but I assume you don't want to do that.

And here is a sample usage:

static void Test(IEnumerable<dynamic> source)
{
    var stream = new MemoryStream();

    var sourceSet = new DapperResultSet(source);
    var serializer = new XmlSerializer(typeof(DapperResultSet));
    serializer.Serialize(stream, sourceSet);

    stream.Position = 0;
    var reader = new StreamReader(stream);
    var xml = reader.ReadToEnd();

    stream.Position = 0;
    var deserializer = new XmlSerializer(typeof(DapperResultSet));
    var target = ((DapperResultSet)deserializer.Deserialize(stream)).Items;
}