How to serialize complex IEnumerable class in protobuf?

Question

I have a RoaringBitmap which represent billions of bits. It is bitmap which is faster/more compact version of BitArray and is unmanaged. It does implement IEnumerable, but more like extension to get positions, and I don't intend to copy their entire repo to fix implementation. In compact form it takes about 100 bytes , in extended through IEnumerable - 1 GB .

I tried using converters with surrogates, but get exception:

public static class ProtobufSerializer
{
    [ProtoContract]
    public sealed class RoaringBitmapSurrogate
    {
        [ProtoMember(1, OverwriteList = true)]
        public byte[] Data { get; set; }

        private const SerializationFormat Format = SerializationFormat.Portable;

        [ProtoConverter]
        public static RoaringBitmapSurrogate Convert(RoaringBitmap obj)
        {
            if (obj == null)
                return null;
            return new RoaringBitmapSurrogate { Data = obj.Serialize(Format) };
        }
        [ProtoConverter]
        public static RoaringBitmap Convert(RoaringBitmapSurrogate obj)
        {
            if(obj == null)
                return null;
            return RoaringBitmap.Deserialize(obj.Data, Format);
        }
    }

    static ProtobufSerializer()
    {
        var model = ProtoBuf.Meta.RuntimeTypeModel.Default;
        //model.Add(typeof(RoaringBitmapSurrogate), true);
        model.Add(typeof(RoaringBitmap), false).SetSurrogate(typeof(RoaringBitmapSurrogate));
    }


    public static byte[] Serialize<T>(T obj)
    {
        var ms = new MemoryStream();
        Serializer.Serialize(ms, obj);
        return ms.ToArray();
    }

    public static T Deserialize<T>(byte[] data)
    {
        return Serializer.Deserialize<T>(new MemoryStream(data));
    }
}

System.InvalidOperationException : For repeated data declared as CRoaring.RoaringBitmap, the underlying collection (CRoaring.RoaringBitmap) must implement ICollection and must not declare itself read-only; alternative (more exotic) collections can be used, but must be declared using their well-known form (for example, a member could be declared as ImmutableHashSet)

How to serialize it in protobuf-net ? Obviously serializing IEnumerable is just stupid. And Im concerned about logic behind IEnumerable serialization overall, because it potentially can be infinite, be a generator or simply too big (like in my case).

Answer 1

Found a solution, just add this to model:

        var bmModel = model.Add(typeof(RoaringBitmap), false);
        bmModel.IgnoreListHandling = true;
        bmModel.SetSurrogate(typeof(RoaringBitmapSurrogate));

Update

Following @MarcGravell suggestion, I ended up with using ISerializer interface (it still complains about IEnumerable so I set the flag to ignore this behavior, but there is no surrogate class anymore):

public static class ProtobufSerializer
{
    private sealed class RoaringBitmapSerializer : ISerializer<RoaringBitmap>
    {
        private const int MaxRoaringBitmapSize = 8 * 1024;//suffice for my needs, but not yours.
        private static readonly ArrayPool<byte> Pool = ArrayPool<byte>.Shared;

        private const SerializationFormat Format = SerializationFormat.Portable;

        public RoaringBitmap Read(ref ProtoReader.State state, RoaringBitmap value)
        {
            var buffer = Pool.Rent(MaxRoaringBitmapSize);
            try
            {
                var read = state.ReadBytes(buffer);
                return RoaringBitmap.Deserialize(read.ToArray(), Format);
            }
            finally
            {
                Pool.Return(buffer);
            }
        }

        public void Write(ref ProtoWriter.State state, RoaringBitmap value)
        {
            state.WriteBytes(value.Serialize(Format));
        }

        public SerializerFeatures Features => SerializerFeatures.CategoryScalar | SerializerFeatures.WireTypeString;
    }

    static ProtobufSerializer()
    {
        var model = ProtoBuf.Meta.RuntimeTypeModel.Default;
        var bmModel = model.Add(typeof(RoaringBitmap), false);
        bmModel.SerializerType = typeof(RoaringBitmapSerializer);
        bmModel.IgnoreListHandling = true;
    }


    public static byte[] Serialize<T>(T obj)
    {
        var ms = new MemoryStream();
        Serializer.Serialize(ms, obj);
        return ms.ToArray();
    }

    public static T Deserialize<T>(byte[] data)
    {
        return Serializer.Deserialize<T>(new MemoryStream(data));
    }
}