Is there any way to directly map a raw array to an image in WPF?

Question

I have a simple path tracer I've written that I'd like to be able to show a real(ish)-time preview of in my GUI. Currently, I have this working with some hacky solutions:

App is .NET Core 3 based. GUI is .NET Core 3 WPF. The GUI is its own project, while the actual renderer is a separate class library. Currently, I'm using an Image object in the GUI, with its source bound to Renderer.ImageBuffer, which is a Bitmap. I've added some INotifyPropertyChanged code to let the GUI know that it should refresh with a user-set interval. I'm using this converter to get something displayed there:

public class ConvertBitmapToBitmapImage : IValueConverter
{
    public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
    {
        if (!(value is System.Drawing.Bitmap rawBitmap)) return null;
        using var memory = new MemoryStream();
        rawBitmap.Save(memory, System.Drawing.Imaging.ImageFormat.Bmp);
        memory.Position = 0;
        var bitmapImage = new BitmapImage();
        bitmapImage.BeginInit();
        bitmapImage.StreamSource = memory;
        bitmapImage.CacheOption = BitmapCacheOption.OnLoad;
        bitmapImage.EndInit();

        return bitmapImage;
    }

    public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
    {
        throw new NotImplementedException();
    }
}

However, when I enable the update code in the renderer, there's a huge amount of overhead converting from the raw bitmap data to the GUI-friendly BitmapImage format, and it slows the render down substantially. As far as I can tell, there's no way to access the BitmapImage object type directly from the backend without making the class library a WPF project itself in Core 3. Is there a better way that I can be doing this, or a more appropriate control for mapping raw values to something viewable in the GUI?

Answer 1

It is a fact of life that you will, sometimes, have to "interop" with System.Drawing.Bitmap in WPF Applications. You are using a BitmapImage because you would like a BitmapSource , right? You are in luck! WriteableBitmap is supported in .NET Core 3.

Your converting in an IValueConverter instance makes this quite tricky. In any case, the idea is to keep only one WriteableBitmap around and use WritePixels properly , to copy the data without the overhead of re-creating the bitmap. WPF will update at each rendering pass, so you will definitely get your result. I suggest finding a way to "stuff" the WriteableBitmap into your view-model and pass it directly as such, without the converter.

You may (possibly will ) have to re-create the source occasionally, because I suspect you might want to allow resizing your application . Whenever the GUI is resized, you will need to new the WriteableBitmap with the new size. You will also have to make sure you get the DPI setting right, if you are developing a DPI-aware application. Thankfully, the WriteableBitmap can manage some of this (note the constructor arguments) for you.

Update

I ran this code (similar to yours)

    //Load a bitmap in memory from file. Size is 1896 x 745, 
    System.Drawing.Bitmap bmp = new System.Drawing.Bitmap(@"...");

    //Start a stopwatch to measure performance.
    Stopwatch watch = new System.Diagnostics.Stopwatch();
    watch.Start();

    //Perform 500 times.
    for (int i = 0; i < 500; i++)
    {
        using (var memory = new MemoryStream())
        {
            bmp.Save(memory, System.Drawing.Imaging.ImageFormat.Bmp);
            memory.Position = 0;

            var bitmapImage = new BitmapImage();
            bitmapImage.BeginInit();
            bitmapImage.StreamSource = memory;
            bitmapImage.CacheOption = BitmapCacheOption.OnLoad;
            bitmapImage.EndInit();

            //testImage is an Image control in the UI.
            testImage.Source = bitmapImage;
        }
    }

    watch.Stop();
    System.Diagnostics.Debug.WriteLine(watch.ElapsedMilliseconds + " ms.");

    //Just one test run of this takes about 7000 ms!!!

This takes ~7 seconds to run.

Then, I run this code:

    //Load a bitmap in memory from file. Size is 1896 x 745, 
    System.Drawing.Bitmap bmp = new System.Drawing.Bitmap(@"...");

    //Create the BitmapSource (a WriteableBitmap). 96 is the dpi setting.
    WriteableBitmap writeableBMP =
        new WriteableBitmap(1896, 745, 96, 96, PixelFormats.Pbgra32, null);

    //Start a stopwatch to measure performance.
    Stopwatch watch = new System.Diagnostics.Stopwatch();
    watch.Start();

    //Perform 500 times.
    for (int i = 0; i < 500; i++)
    {
        System.Drawing.Rectangle rect = new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height);

        //Lock the bits to copy from the bitmap to the image source.
        BitmapData data =
            bmp.LockBits(rect, ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);

        //Prepare the buffer size (System.Drawing.Bitmap specific)
        int bufferSize = rect.Height * data.Stride;

        //Prepare the write rectangle, of course this is the entire image.
        Int32Rect writeRectangle = new Int32Rect(0, 0, 1896, 745);

        //Write the pixels
        writeableBMP.WritePixels(writeRectangle, data.Scan0, bufferSize, data.Stride, 0, 0);

        //Unlock the bits, to prepare for another cycle.
        bmp.UnlockBits(data);

        testImage.Source = writeableBMP;   
    }

    watch.Stop();
    System.Diagnostics.Debug.WriteLine(watch.ElapsedMilliseconds + " ms.");

    //Multiple test runs of this take ~300 ms!!!

This second piece of code, in all its glorious sloppiness, executes in ~300 ms. It copies the entire bitmap in every step . A 1896 x 745 bitmap, should be relatively average for a screen UI that is "almost" maximized.

My suggestion to use a WriteableBitmap improves the run-time of achieving the same result with the originally posted code by about ~7000/300 > ~20 times in my machine. Would you possibly try this as I have posted it, and let me know if it runs any better for you? Having had exactly the same problem with you (and switching to a WriteableBitmap of course), I am hard pressed to believe that it only improves your running time by 1-2% . You may have to share some more details of your UI rendering code, so that I can try to locate other potential bottlenecks.

Of course some overhead is to be expected by the rest of the WPF rendering pipeline, as I am only measuring a critical part of the code (relatively roughly, no less) but, in general, the WriteableBitmap is close to "as good as it gets" before you abandon software rendering in WPF for any other rendering engine (eg hardware).

(I have received 1 downvote and 1 vote to delete this answer so far)

Answer 2

You might want to use BitmapSource.Create :

int GetStride(PixelFormat pixelFormat, int width)
{
    var bytesPerPixel = (pixelFormat.BitsPerPixel + 7) / 8;
    return bytesPerPixel * width;
}

var dpi = 96;
var height = 1080;
var width = 2160;
var size = width * height * 3;
var pixelFmt = PixelFormats.Bgr24;
var stride = GetStride(pixelFmt, width);
var pixelData = new byte[size];
var bmpSource = BitmapSource.Create(width, height, dpi, dpi, pixelFmt, null,
                                    pixelData, stride);

https://docs.microsoft.com/en-us/dotnet/api/system.windows.media.imaging.bitmapsource.create?view=netcore-3.0