Formatting byte[] or ReadOnlySpan<byte> to string using C# 10 and .NET 6 string interpolation, compiler handler lowering pattern

Question

I would like to format byte[] and ReadOnlySpan<byte> bytes to strings using a few, custom formatting parameters. Say, like S for Base64 . For the purpose of this, the length is always fixed to some known constant.

I would like to use the modern C# 10 and .NET 6 string formatting capabilities as described at https://devblogs.microsoft.com/dotnet/string-interpolation-in-c-10-and-net-6/ . The built-in types implement ISpanFormattable and so what I'd like to bring here is new formatting parameters but so that compiler handler lowering pattern is used.

I took some code from that post and modified it a bit in the code embedded as follows. It's also at https://dotnetfiddle.net/svyQKD .

As is seen in the code, I get the direct method call to succeed for byte[] , but not for ReadOnlySpan<byte> .

Does anyone have a clue how to do that?

I suspect I need InterpolatedStringHandler . But if that is the case, then it looks like I don't know how to implement one. All tips and code tricks would probably help. I've been stuck at this for a while now and it's getting into wee hours. :)

using System;
using System.Globalization;
using System.Runtime.CompilerServices;

public class Program
{
    public sealed class ExampleCustomFormatter: IFormatProvider, ICustomFormatter
    {
        public object? GetFormat(Type? formatType) => formatType == typeof(ICustomFormatter) ? this : null;
        public string Format(string? format, object? arg, IFormatProvider? formatProvider) => format == "S" && arg is byte[] i ? Convert.ToBase64String(i) : arg is IFormattable formattable ? formattable.ToString(format, formatProvider) : arg?.ToString() ?? string.Empty;
    }

    public static class StringExtensions
    {
        public static string FormatString(byte[] buffer) => string.Create(new ExampleCustomFormatter(), stackalloc char[64], $"{buffer:S}");

        // How to make this work? Maybe needs to have TryWrite 
        // public static string FormatString2(ReadOnlySpan<byte> buffer) => string.Create(new ExampleCustomFormatter(), stackalloc char[64], $"{buffer:S}");
    }

    [InterpolatedStringHandler]
    public ref struct BinaryMessageInterpolatedStringHandler
    {
        private readonly DefaultInterpolatedStringHandler handler;

        public BinaryMessageInterpolatedStringHandler(int literalLength, int formattedCount, bool predicate, out bool handlerIsValid)
        {
            handler = default;

            if(predicate)
            {
                handlerIsValid = false;
                return;
            }

            handlerIsValid = true;
            handler = new DefaultInterpolatedStringHandler(literalLength, formattedCount);
        }

        public void AppendLiteral(string s) => handler.AppendLiteral(s);

        public void AppendFormatted<T>(T t) => handler.AppendFormatted(t);

        public override string ToString() => handler.ToStringAndClear();
    }


    public static void Main()
    {
        byte[] test1 = new byte[1] { 0x55 };
        ReadOnlySpan<byte> test2 = new byte[1] { 0x55 };

        // How to make this work? Now it prints "System.Byte[]".
        Console.WriteLine($"{test1:S}");

        // This works.
        Console.WriteLine(StringExtensions.FormatString(test1));

        // How to make this work? This does not compile. (Yes, signature problem. How to define it?).
        // Console.WriteLine($"{test2:S}");

        // How to make this work? This does not compile. (Yes, signature problem. How to define it?).

        // Console.WriteLine(StringExtensions.FormatString(test2));
    }
}

Answer 1

If you really want to use the method like this, you need to override the ToString() method of the class Byte[] .

But you cannot override the method on the very class Byte[] . You need to inherit the class Byte[] and override the ToString() method on the derived.

Then, you must replace all your Byte[] objects with the derived class, with is not a good idea.

So, there is no solution for you in this manner:

// How to make this work? Now it prints "System.Byte[]".
Console.WriteLine($"{test1:S}");

The best you can do is create an "outside" method for formatting the Byte[] and do as you with for formatting.

*The same applies to the ReadOnlySpan<byte> .

Answer 2

You can use an extension method:

using System.Text;

byte[] test1 = new byte[2] { 0x55, 0x34 };
ReadOnlySpan<byte> test2 = new byte[2] { 0x55, 0x34 };

// How to make this work? Now it prints "System.Byte[]".
Console.WriteLine($"{test1.MyFormat()}");

Console.WriteLine($"{test2.MyFormat()}");

public static class MyExtensionMethods
{
    public static string MyFormat(this byte[] value)
    {
        StringBuilder sb = new StringBuilder();
        foreach (byte b in value)
        {
            sb.Append(b).Append(" ");
        }
        return sb.ToString();
    }

    public static string MyFormat(this ReadOnlySpan<byte> value)
    {
        StringBuilder sb = new StringBuilder();
        foreach (byte b in value)
        {
            sb.Append(b).Append(" ");
        }
        return sb.ToString();
    }
}

Result:

85 52
85 52

You can try also with:

public static class MyExtensionMethods
{
    public static string MyFormat(this byte[] value) => Encoding.Unicode.GetString(value);

    public static string MyFormat(this ReadOnlySpan<byte> value) => Encoding.Unicode.GetString(value);
}

Answer 3

I played around with this as I was also interested in learning about these handlers, so consider my answer with some suspicion.

The build error CS0306 The type 'ReadOnlySpan<byte>' may not be used as a type argument on Console.WriteLine($"{test2:S}"); (and StringExtensions.FormatString2 ) stem from (I believe) the compiler-generated call to DefaultInterpolatedStringHandler.AppendFormatted<T> with generic type argument ReadOnlySpan<byte> which is illegal ( ReadOnlySpan is a ref struct and a ref struct cannot be used as a type argument).

As explained in Stephen Toub's blog which you linked, the compiler will happily expand any interpolated string to use DefaultInterpolatedStringHandler (if it so desires) which is why you may use an interpolated string with Console.WriteLine even if it does not have an overload taking a DefaultInterpolatedStringHandler parameter:

// This trivial example might not actually construct a DefaultInterpolatedStringHandler
// but the point still stands (it compiles and works)
int x = 3;
Console.WriteLine($"{x}");

An easy solution would have been to create an extension overload of DefaultInterpolatedStringHandler.AppendFormatted taking a ReadOnlySpan<byte> but in this cruel world the compiler does not appear to detect it:

public static class PleaseWork
{
    public static void AppendFormatted(
        this ref DefaultInterpolatedStringHandler handler,
        ReadOnlySpan<byte> bytes,
        string? format)
    {
        if (format != "S")
        {
            throw new ArgumentException("Invalid format");
        }

        handler.AppendLiteral(Convert.ToBase64String(bytes));
    }
}

ReadOnlySpan<byte> test2 = new byte[1] { 0x56 };
Console.WriteLine($"{test2:S}"); // nope (CS0306)

So here is a solution: create a custom InterpolatedStringHandler (as you have done) with the AppendFormatted overload. Like in your implementation we can just wrap a DefaultInterpolatedStringHandler (or a StringBuilder ) so that the actual implementation (buffer management etc) is left to the more clever folk. Full example:

public class Program
{
    [InterpolatedStringHandler]
    public ref struct WrappingInterpolatedStringHandler
    {
        private DefaultInterpolatedStringHandler handler;

        public WrappingInterpolatedStringHandler(int literalLength, int formattedCount)
        {
            handler = new(literalLength, formattedCount);
        }

        public void AppendLiteral(string s) => handler.AppendLiteral(s);

        public void AppendFormatted<T>(T t, string? format = null)
            => handler.AppendFormatted(t, format);

        // This is necessary otherwise byte[] arg resolves to the generic overload
        public void AppendFormatted(byte[] bytes, string? format = null)
            => AppendFormatted((ReadOnlySpan<byte>)bytes, format);

        public void AppendFormatted(ReadOnlySpan<byte> bytes, string? format)
        {
            if (format != "S")
            {
                throw new ArgumentException("Invalid format");
            }
            
            // This allocates an intermediate string
            handler.AppendLiteral(Convert.ToBase64String(bytes));
        }

        public string ToStringAndClear() => handler.ToStringAndClear();
    }

    public static void Main()
    {
        byte[] test1 = new byte[1] { 0x55 };
        ReadOnlySpan<byte> test2 = new byte[1] { 0x56 };
        Span<byte> test3 = new byte[128];
        test3.Fill(0x55);

        WriteLine($"test1:{test1:S}"); // "test1:VQ=="
        WriteLine($"test2:{test2:S}"); // "test2:Vg=="
        WriteLine($"test3:{test3:S}"); // "test3:<a long base64 string>"
    }

    private static void WriteLine(ref WrappingInterpolatedStringHandler builder)
    {
        // Naturally, Console.WriteLine has no overload taking our custom handler
        // so we need to give it the resulting string
        Console.WriteLine(builder.ToStringAndClear());
    }
}

Our new AppendFormatted overload calls Convert.ToBase64String which is allocating an intermediate string (which is then copied into the buffer of the DefaultInterpolatedStringHandler ). That's the (vanishingly small?) price we pay for not managing the buffer ourselves. If we couldn't bear the thought (to hell with implementation risk and diminishing returns), here is a dumb handler taking a (fixed size) buffer as an argument (which can be stack-allocted):

public class Program
{
    [InterpolatedStringHandler]
    public ref struct MyReallyBadAndUntestedInterpolatedStringHandler
    {
        private readonly Span<char> _buffer;
        private int _charsWritten;

        public MyReallyBadAndUntestedInterpolatedStringHandler(int literalLength, int formattedCount, Span<char> destination)
        {
            _buffer = destination;
            _charsWritten = 0;
        }

        public bool AppendLiteral(string s)
        {
            if (s.AsSpan().TryCopyTo(_buffer.Slice(_charsWritten)))
            {
                _charsWritten += s.Length;
                return true;
            }

            return false;
        }

        public bool AppendFormatted<T>(T t, ReadOnlySpan<char> format)
        {
            Span<char> buffer = _buffer.Slice(_charsWritten);
            
            if (t is ISpanFormattable formattable)
            {
                bool success = formattable.TryFormat(buffer, out int charsWritten, format, null);
                _charsWritten += _charsWritten;
                return success;
            }

            string s = t?.ToString() ?? "";

            if (s.AsSpan().TryCopyTo(buffer))
            {
                _charsWritten += s.Length;
                return true;
            }

            return false;
        }

        // This is necessary otherwise byte[] arg resolves to the generic overload
        public bool AppendFormatted(byte[] bytes, ReadOnlySpan<char> format)
            => AppendFormatted((ReadOnlySpan<byte>)bytes, format);

        public bool AppendFormatted(ReadOnlySpan<byte> bytes, ReadOnlySpan<char> format)
        {
            if (format != "S")
            {
                throw new ArgumentException("Invalid format");
            }

            bool success = Convert.TryToBase64Chars(bytes, _buffer.Slice(_charsWritten), out int charsWritten);
            _charsWritten += charsWritten;
            return success;
        }

        public string ToStringAndClear()
        {
            string result = new string(_buffer.Slice(0, _charsWritten));
            _buffer.Clear();
            _charsWritten = 0;
            return result;
        }
    }

    public static void Main()
    {
        byte[] test1 = new byte[1] { 0x55 };
        ReadOnlySpan<byte> test2 = new byte[1] { 0x56 };
        Span<byte> test3 = new byte[128];
        test3.Fill(0x55);

        Span<char> buffer = stackalloc char[64];
        WriteLine(buffer, $"test1:{test1:S}"); // "test1:VQ=="
        WriteLine(buffer, $"test2:{test2:S}"); // "test2:Vg=="
        WriteLine(buffer, $"test3:{test3:S}"); // "test3:" (the buffer is not big enough)
    }

    private static void WriteLine(
        Span<char> destination,
        [InterpolatedStringHandlerArgument("destination")] ref MyReallyBadAndUntestedInterpolatedStringHandler builder)
    {
        Console.WriteLine(builder.ToStringAndClear());
    }
}

In conclusion just call Console.WriteLine(Convert.ToBase64String(test2)) and move on:)