Just serialise to byte[] Protobuf

Question

I'm trying to write a benchmark that compares JSON serialisation to Protobuf serialisation speed for a demo I'm doing tomorrow.

using System.Text;
using System.Text.Json;
using AutoFixture;
using BenchmarkDotNet.Attributes;
using ProtobufDemo.ProtoModels;
using Google.Protobuf;

namespace Benchmark;

public class JsonVsProtoBenchmark
{
    private readonly AddressBook _addressBook;

    public JsonVsProtoBenchmark()
    {
        Fixture fixture = new();
        _addressBook = fixture.Create<AddressBook>();
    }

    [Benchmark]
    public void SerialiseToJsonOnly() => JsonSerializer.Serialize(_addressBook);

    [Benchmark]
    public void SerialiseToJsonEncodeAndWrite() // This is more representative of what the protobuf serialiser is actually doing
    {
        string json = JsonSerializer.Serialize(_addressBook);
        byte[] encodedBytes = Encoding.UTF8.GetBytes(json);
        Stream.Null.Write(encodedBytes);
        Stream.Null.Flush();
    }

    [Benchmark]
    public void SerialiseToProtobufAndWrite() => _addressBook.WriteTo(Stream.Null); // Note we have an extra write here that's not necessarily fair
}

However, this is far from a fair test as IMessage.WriteTo(Stream) does far more than just the serialisation. Even the first layer down does loads:

public static void WriteTo(this IMessage message, Stream output)
{
    ProtoPreconditions.CheckNotNull(message, "message");
    ProtoPreconditions.CheckNotNull(output, "output");
    CodedOutputStream codedOutput = new CodedOutputStream(output);
    message.WriteTo(codedOutput);
    codedOutput.Flush();
}

I've tried to compensate for some of this with my SerialiseToJsonEncodeAndWrite() test, but even then, it's not really an apples to apples comparison.

Is there any way I can just perform the serialisation step rather than going through .WriteTo() .

Answer 1

It turns out that I can avoid using the extension method that accepts a Stream and instead use the .WriteTo(CodedOutputStream) method directly, which removes almost all of this overhead. If I pre-create this in my benchmark setup along with an instance of the NullStream I get much more representative tests.

public class JsonVsProtoBenchmark
{
    private AddressBook _addressBook = null!;
    private CodedOutputStream _codedOutputStream = null!;
    private Stream _nullStream = null!;

    [GlobalSetup]
    public void Setup()
    {
        Fixture fixture = new();
        _addressBook = fixture.Create<AddressBook>();
        _codedOutputStream = new(Stream.Null);
        _nullStream = Stream.Null;
    }

    [Benchmark]
    public void SerialiseToSystemTextJsonOnly() => JsonSerializer.Serialize(_addressBook);

    [Benchmark]
    public void SerialiseToJsonEncodeAndWrite() // This is more representative of what the protobuf serialiser is actually doing
    {
        string json = JsonSerializer.Serialize(_addressBook);
        byte[] encodedBytes = Encoding.UTF8.GetBytes(json);
        _nullStream.Write(encodedBytes);
        _nullStream.Flush();
    }

    [Benchmark]
    public void SerialiseToProtobufAndWrite() => _addressBook.WriteTo(_codedOutputStream); // Note we have an extra write here that's not necessarily fair
}

There are clearly still some issues with this, but it seems far closer than when using the extension method.