C# SkiaSharp OpenTK Winform - 如何从后台线程绘制？

Question

I am trying to replace GDI+ with SkiaSharp for a data visualization framework that renders multi-layered pannable-zoomable graphs with real-time continuously changing engineering data.

In GDI+, the application did this:

• Created a collection of drawing layers with transparent backgrounds, typically a Grid Layer, one or more Data Layers, and an Overlay Layer for cursor info and highlighting, each backed by a separate bitmap.
• In a rendering loop background thread, only the layers (bitmaps) that needed to be updated for each rendering cycle were redrawn using GDI+. This could require many thousands of calculated and transformed lines, rectangles, and text to create heatmaps, waveforms, histogram plots, data labels, etc.
• Each drawing layer in the stack would then be BitBlt'd to a composite bitmap by the background thread
• The final composite bitmap would then be drawn to a WinForm PictureBox in the GUI thread at up to 30fps.

Everything up to the final image presentation was done in one or more background threads. The GUI thread was only involved to draw the finished image to the PictureBox. This is important because there are many other GUI controls that need to stay responsive. This worked great, except it is all CPU based. Small windows were no problem, but maximizing on a 4K screen would slow down the rendering enough to make the program pretty much unusable.

I would like to recreate this concept with GPU accelerated SkiaSharp.

I tried creating dozens of different test programs and I keep getting Cross-Thread access violations, or nothing showing on the screen, or hard crashes. Instead of posting code, let me ask some basic questions:

Questions:

• How would you approach creating this framework ? Can SkiaSharp even do this ?
• Should each of my layer classes maintain an SKSurface, SKCanvas, SKImage, or SKBitmap ? - Again, if a layer does not need to be redrawn for the current cycle, then the layer needs to maintain the previously drawn content for use in the next composite image.
• A GLControl and GRContext is needed on the GUI thread for showing the final composite image, but should there be another separate GRContext for the background rendering thread to use ? - How to create with GPU acceleration ?
• Are there any working examples of a similar concept that someone can point to ? ( GPU accelerating rendering from background thread to GLControl )
• Should I use SkiaSharp hidden in the background only, and GDI+ BitBlt with PictureBox to show the composite image on the screen ? - Would that solve some threading issues ?

Any help defining the approach and the dos and don'ts would be greatly appreciated !!

Answer 1

I figured out how to get this to work using SKPicture objects to record the Draw Commands from each layer using a background rendering thread, then painting them back to an SKGLControl using the GUI thread. This satisfies all of my requirements: It allows for multiple drawing layers, renders with a background thread, renders only the layers that need updates, paints with GPU acceleration, and is extremely fast for a maximized 4K window.

Lessons Learned

There are a few lessons that I learned along the way that were causing a lot of confusion for me...

1. There are examples online of using an OpenTK.GLControl with GPU acceleration, and there are examples using the SkiaSharp.Views.Desktop.SKGLControl which has built in GPU acceleration. The SKGLControl is definitely the correct control for this task. The GLControl was creating squares for DrawCircle and refusing to render any curves due to issues with the FramebufferBinding and StencilBits ?!? - I gave up on it. It is also slower than the SKGLControl for SKPicture objects.

2. The SKGLControl does not need nor like the use of SwapBuffers or Canvas.Flush, which are required for the GLControl. This was causing strobing and glitching of the drawings for SKGLControl, which is why I went off in the weeds fighting with the GLControl. When I rebuilt the project with SKGLControl and got rid of SwapBuffers and Canvas.Flush, everything started behaving.

3. References to Surfaces and Canvases should not be held past one PaintSurface cycle. The SKPicture is the magical object that will let you store the drawing commands for each layer and play them back again and again. This is different from an SKBitmap or SKImage which are generating pixel rasters instead of just recording the Draw commands. I couldn't get SKBitmap or SKImage to behave in a multithreaded environment and still be GPU accelerated. SKPicture works great for this.

4. There is a difference between the Paint event and the PaintSurface event for the SKGLControl. The PaintSurface event is what should be used and is by default GPU accelerated.

---

Working Example Code

Below is fully functional demo of a multi-layer, multi-threaded, GPU accelerated SkiaSharp drawing

This example creates 4 drawing layers:

• Background Layer
• Grid Layer
• Data Layer
• Overlay Layer

The layers are drawn ( rendered ) using a background thread, then painted to an SKGLControl using the GUI thread. Each layer is only rendered when needed, but all layers are painted with each PaintSurface event.

To try the code:

1. Create a new C# WinForms project in Visual Studio.
2. Add the NuGet Package: "SkiaSharp.Views.WindowsForms". This will automatically add "SkiaSharp", and "SkiaSharp.Views.Desktop.Common".
3. Add a SkiaSharp.Views.Desktop.SKGLControl to Form1. Name it "skglControl1"
4. Set Dock to "Fill" for skglControl1 so that it fills Form1.
5. Copy the code below to Form1:

    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.Drawing;
    using System.Threading;
    using System.Windows.Forms;
    using SkiaSharp;
    using SkiaSharp.Views.Desktop;
    
    namespace SkiaSharp_Multi_Layer_GPU
    {
        // ---------------------------------------------------------------------
        // ---------------------------------------------------------------------
        // -------                                                       -------
        // -------                   WinForm - Form 1                    -------
        // -------                                                       -------
        // ---------------------------------------------------------------------
        // ---------------------------------------------------------------------

        public partial class Form1 : Form
        {
            private Thread m_RenderThread = null;
            private AutoResetEvent m_ThreadGate = null;
            private List<Layer> m_Layers = null;
            private Layer m_Layer_Background = null;
            private Layer m_Layer_Grid = null;
            private Layer m_Layer_Data = null;
            private Layer m_Layer_Overlay = null;
            private bool m_KeepSwimming = true;
            private SKPoint m_MousePos = new SKPoint();
            private bool m_ShowGrid = true;
            private Point m_PrevMouseLoc = new Point();
    
    
            // ---------------------------
            // --- Form1 - Constructor ---
            // ---------------------------
    
            public Form1()
            {
                InitializeComponent();
            }
    
    
            // ------------------------------
            // --- Event - Form1 - OnLoad ---
            // ------------------------------
    
            protected override void OnLoad(EventArgs e)
            {
                base.OnLoad(e);
    
                // Set the title of the Form
                this.Text = "SkiaSharp Demo - Multi-Layer, Multi-Threaded, GPU Accelerated";
    
                // Create layers to draw on, each with a dedicated SKPicture
                m_Layer_Background = new Layer("Background Layer");
                m_Layer_Grid = new Layer("Grid Layer");
                m_Layer_Data = new Layer("Data Layer");
                m_Layer_Overlay = new Layer("Overlay Layer");
    
                // Create a collection for the drawing layers
                m_Layers = new List<Layer>();
                m_Layers.Add(m_Layer_Background);
                m_Layers.Add(m_Layer_Grid);
                m_Layers.Add(m_Layer_Data);
                m_Layers.Add(m_Layer_Overlay);
    
                // Subscribe to the Draw Events for each layer
                m_Layer_Background.Draw += Layer_Background_Draw;
                m_Layer_Grid.Draw += Layer_Grid_Draw;
                m_Layer_Data.Draw += Layer_Data_Draw;
                m_Layer_Overlay.Draw += Layer_Overlay_Draw;
    
                // Subscribe to the SKGLControl events
                skglControl1.PaintSurface += SkglControl1_PaintSurface;
                skglControl1.Resize += SkglControl1_Resize;
                skglControl1.MouseMove += SkglControl1_MouseMove;
                skglControl1.MouseDoubleClick += SkglControl1_MouseDoubleClick;
                
                // Create a background rendering thread
                m_RenderThread = new Thread(RenderLoopMethod);
                m_ThreadGate = new AutoResetEvent(false);
    
                // Start the rendering thread
                m_RenderThread.Start();
            }
    
    
            // ---------------------------------
            // --- Event - Form1 - OnClosing ---
            // ---------------------------------
    
            protected override void OnClosing(CancelEventArgs e)
            {
                // Let the rendering thread terminate
                m_KeepSwimming = false;
                m_ThreadGate.Set();
    
                base.OnClosing(e);
            }
    
    
            // --------------------------------------------
            // --- Event - SkglControl1 - Paint Surface ---
            // --------------------------------------------
    
            private void SkglControl1_PaintSurface(object sender, SkiaSharp.Views.Desktop.SKPaintGLSurfaceEventArgs e)
            {
                // Clear the Canvas
                e.Surface.Canvas.Clear(SKColors.Black);
    
                // Paint each pre-rendered layer onto the Canvas using this GUI thread
                foreach (var layer in m_Layers)
                {
                    layer.Paint(e.Surface.Canvas);
                }
    
    
                using (var paint = new SKPaint())
                {
                    paint.Color = SKColors.LimeGreen;
    
                    for (int i = 0; i < m_Layers.Count; i++)
                    {
                        var layer = m_Layers[i];
                        var text = $"{layer.Title} - Renders = {layer.RenderCount}, Paints = {layer.PaintCount}";
                        var textLoc = new SKPoint(10, 10 + (i * 15));
    
                        e.Surface.Canvas.DrawText(text, textLoc, paint);
                    }
    
    
                    paint.Color = SKColors.Cyan;
    
                    e.Surface.Canvas.DrawText("Click-Drag to update bars.", new SKPoint(10, 80), paint);
                    e.Surface.Canvas.DrawText("Double-Click to show / hide grid.", new SKPoint(10, 95), paint);
                    e.Surface.Canvas.DrawText("Resize to update all.", new SKPoint(10, 110), paint);
                }
            }
    
    
            // -------------------------------------
            // --- Event - SkglControl1 - Resize ---
            // -------------------------------------
    
            private void SkglControl1_Resize(object sender, EventArgs e)
            {
                // Invalidate all of the Layers
                foreach (var layer in m_Layers)
                {
                    layer.Invalidate();
                }
    
                // Start a new rendering cycle to redraw all of the layers.
                UpdateDrawing();
            }
    
    
            // -----------------------------------------
            // --- Event - SkglControl1 - Mouse Move ---
            // -----------------------------------------
    
            private void SkglControl1_MouseMove(object sender, MouseEventArgs e)
            {
                // Save the mouse position
                m_MousePos = e.Location.ToSKPoint();
    
                // If Left-Click Drag, draw new bars
                if (e.Button == MouseButtons.Left)
                {
                    // Invalidate the Data Layer to draw a new random set of bars
                    m_Layer_Data.Invalidate();
                }
    
                // If Mouse Move, draw new mouse coordinates
                if (e.Location != m_PrevMouseLoc)
                {
                    // Remember the previous mouse location
                    m_PrevMouseLoc = e.Location;
    
                    // Invalidate the Overlay Layer to show the new mouse coordinates
                    m_Layer_Overlay.Invalidate();
                }
    
                // Start a new rendering cycle to redraw any invalidated layers.
                UpdateDrawing();
            }
    
    
            // -------------------------------------------------
            // --- Event - SkglControl1 - Mouse Double Click ---
            // -------------------------------------------------
    
            private void SkglControl1_MouseDoubleClick(object sender, MouseEventArgs e)
            {
                // Toggle the grid visibility
                m_ShowGrid = !m_ShowGrid;
    
                // Invalidate only the Grid Layer.  
                m_Layer_Grid.Invalidate();
    
                // Start a new rendering cycle to redraw any invalidated layers.
                UpdateDrawing();
            }
    
    
            // ----------------------
            // --- Update Drawing ---
            // ----------------------
    
            public void UpdateDrawing()
            {
                // Unblock the rendering thread to begin a render cycle.  Only the invalidated
                // Layers will be re-rendered, but all will be repainted onto the SKGLControl.
                m_ThreadGate.Set();
            }
    
    
            // --------------------------
            // --- Render Loop Method ---
            // --------------------------
    
            private void RenderLoopMethod()
            {
                while (m_KeepSwimming)
                {
                    // Draw any invalidated layers using this Render thread
                    DrawLayers();
    
                    // Invalidate the SKGLControl to run the PaintSurface event on the GUI thread
                    // The PaintSurface event will Paint the layer stack to the SKGLControl
                    skglControl1.Invalidate();
    
                    // DoEvents to ensure that the GUI has time to process
                    Application.DoEvents();
    
                    // Block and wait for the next rendering cycle
                    m_ThreadGate.WaitOne();
                }
            }
    
    
            // -------------------
            // --- Draw Layers ---
            // -------------------
    
            private void DrawLayers()
            {
                // Iterate through the collection of layers and raise the Draw event for each layer that is
                // invalidated.  Each event handler will receive a Canvas to draw on along with the Bounds for 
                // the Canvas, and can then draw the contents of that layer. The Draw commands are recorded and  
                // stored in an SKPicture for later playback to the SKGLControl.  This method can be called from
                // any thread.
    
                var clippingBounds = skglControl1.ClientRectangle.ToSKRect();
    
                foreach (var layer in m_Layers)
                {
                    layer.Render(clippingBounds);
                }
            }
    
    
            // -----------------------------------------
            // --- Event - Layer - Background - Draw ---
            // -----------------------------------------
    
            private void Layer_Background_Draw(object sender, EventArgs_Draw e)
            {
                // Create a diagonal gradient fill from Blue to Black to use as the background
                var topLeft = new SKPoint(e.Bounds.Left, e.Bounds.Top);
                var bottomRight = new SKPoint(e.Bounds.Right, e.Bounds.Bottom);
                var gradColors = new SKColor[2] { SKColors.DarkBlue, SKColors.Black };
    
                using (var paint = new SKPaint())
                using (var shader = SKShader.CreateLinearGradient(topLeft, bottomRight, gradColors, SKShaderTileMode.Clamp))
                {
                    paint.Shader = shader;
                    paint.Style = SKPaintStyle.Fill;
                    e.Canvas.DrawRect(e.Bounds, paint);
                }
            }
    
    
            // -----------------------------------
            // --- Event - Layer - Grid - Draw ---
            // -----------------------------------
    
            private void Layer_Grid_Draw(object sender, EventArgs_Draw e)
            {
                if (m_ShowGrid)
                {
                    // Draw a 25x25 grid of gray lines
    
                    using (var paint = new SKPaint())
                    {
                        paint.Color = new SKColor(64, 64, 64); // Very dark gray
                        paint.Style = SKPaintStyle.Stroke;
                        paint.StrokeWidth = 1;
    
                        // Draw the Horizontal Grid Lines
                        for (int i = 0; i < 50; i++)
                        {
                            var y = e.Bounds.Height * (i / 25f);
                            var leftPoint = new SKPoint(e.Bounds.Left, y);
                            var rightPoint = new SKPoint(e.Bounds.Right, y);
    
                            e.Canvas.DrawLine(leftPoint, rightPoint, paint);
                        }
    
                        // Draw the Vertical Grid Lines
                        for (int i = 0; i < 50; i++)
                        {
                            var x = e.Bounds.Width * (i / 25f);
                            var topPoint = new SKPoint(x, e.Bounds.Top);
                            var bottomPoint = new SKPoint(x, e.Bounds.Bottom);
    
                            e.Canvas.DrawLine(topPoint, bottomPoint, paint);
                        }
                    }
                }
            }
    
    
            // -----------------------------------
            // --- Event - Layer - Date - Draw ---
            // -----------------------------------
    
            private void Layer_Data_Draw(object sender, EventArgs_Draw e)
            {
                // Draw a simple bar graph
    
                // Flip the Y-Axis so that zero is on the bottom
                e.Canvas.Scale(1, -1);
                e.Canvas.Translate(0, -e.Bounds.Height);
    
                var rand = new Random();
    
                // Create 25 red / yellow gradient bars of random length
                for (int i = 0; i < 25; i++)
                {
                    var barWidth = e.Bounds.Width / 25f;
                    var barHeight = rand.Next((int)(e.Bounds.Height * 0.65d));
                    var barLeft = (i + 0) * barWidth;
                    var barRight = (i + 1) * barWidth;
                    var barTop = barHeight;
                    var barBottom = 0;
                    var topLeft = new SKPoint(barLeft, barTop);
                    var bottomRight = new SKPoint(barRight, barBottom);
                    var gradColors = new SKColor[2] { SKColors.Yellow, SKColors.Red };
    
                    // Draw each bar with a gradient fill
                    using (var paint = new SKPaint())
                    using (var shader = SKShader.CreateLinearGradient(topLeft, bottomRight, gradColors, SKShaderTileMode.Clamp))
                    {
                        paint.Style = SKPaintStyle.Fill;
                        paint.StrokeWidth = 1;
                        paint.Shader = shader;
    
                        e.Canvas.DrawRect(barLeft, barBottom, barWidth, barHeight, paint);
                    }
    
                    // Draw the border of each bar
                    using (var paint = new SKPaint())
                    {
                        paint.Color = SKColors.Blue;
                        paint.Style = SKPaintStyle.Stroke;
                        paint.StrokeWidth = 1;
    
                        e.Canvas.DrawRect(barLeft, barBottom, barWidth, barHeight, paint);
                    }
                }
            }
    
    
            // --------------------------------------
            // --- Event - Layer - Overlay - Draw ---
            // --------------------------------------
    
            private void Layer_Overlay_Draw(object sender, EventArgs_Draw e)
            {
                // Draw the mouse coordinate text next to the cursor
    
                using (var paint = new SKPaint())
                {
                    // Configure the Paint to draw a black rectangle behind the text
                    paint.Color = SKColors.Black;
                    paint.Style = SKPaintStyle.Fill;
    
                    // Measure the bounds of the text
                    var text = m_MousePos.ToString();
                    SKRect textBounds = new SKRect();
                    paint.MeasureText(text, ref textBounds);
    
                    // Fix the inverted height value from the MeaureText
                    textBounds = textBounds.Standardized;
                    textBounds.Location = new SKPoint(m_MousePos.X, m_MousePos.Y - textBounds.Height);
    
                    // Draw the black filled rectangle where the text will go
                    e.Canvas.DrawRect(textBounds, paint);
    
                    // Change the Paint to yellow
                    paint.Color = SKColors.Yellow;
    
                    // Draw the mouse coordinates text
                    e.Canvas.DrawText(m_MousePos.ToString(), m_MousePos, paint);
                }
            }
        }
    
    
    
        // ---------------------------------------------------------------------
        // ---------------------------------------------------------------------
        // -------                                                       -------
        // -------                     Class - Layer                     -------
        // -------                                                       -------
        // ---------------------------------------------------------------------
        // ---------------------------------------------------------------------
    
        public class Layer
        {
            // The Draw event that the background rendering thread will use to draw on the SKPicture Canvas.  
            public event EventHandler<EventArgs_Draw> Draw;
    
            // The finished recording - Used to play back the Draw commands to the SKGLControl from the GUI thread
            private SKPicture m_Picture = null;
    
            // A flag that indicates if the Layer is valid, or needs to be redrawn.
            private bool m_IsValid = false;
    
    
            // ---------------------------
            // --- Layer - Constructor ---
            // ---------------------------
    
            public Layer(string title)
            {
                this.Title = title;
            }
    
    
            // -------------
            // --- Title ---
            // -------------
    
            public string Title { get; set; }
    
    
            // --------------
            // --- Render ---
            // --------------
    
     
            // Raises the Draw event and records any drawing commands to an SKPicture for later playback.  
            // This can be called from any thread.
      
    
            public void Render(SKRect clippingBounds)
            {
                // Only redraw the Layer if it has been invalidated
                if (!m_IsValid)
                {
                    // Create an SKPictureRecorder to record the Canvas Draw commands to an SKPicture
                    using (var recorder = new SKPictureRecorder())
                    {
                        // Start recording 
                        recorder.BeginRecording(clippingBounds);
    
                        // Raise the Draw event.  The subscriber can then draw on the Canvas provided in the event
                        // and the commands will be recorded for later playback.
                        Draw?.Invoke(this, new EventArgs_Draw(recorder.RecordingCanvas, clippingBounds));
    
                        // Dispose of any previous Pictures
                        m_Picture?.Dispose();
    
                        // Create a new SKPicture with recorded Draw commands 
                        m_Picture = recorder.EndRecording();
    
                        this.RenderCount++;
    
                        m_IsValid = true;
                    }
                }
            }
    
    
            // --------------------
            // --- Render Count ---
            // --------------------
    
            // Gets the number of times that this Layer has been rendered
    
            public int RenderCount { get; private set; }
    
    
            // -------------
            // --- Paint ---
            // -------------
    
            // Paints the previously recorded SKPicture to the provided skglControlCanvas.  This basically plays 
            // back the draw commands from the last Render.  This should be called from the SKGLControl.PaintSurface
            // event using the GUI thread.
    
            public void Paint(SKCanvas skglControlCanvas)
            {
                if (m_Picture != null)
                {
                    // Play back the previously recorded Draw commands to the skglControlCanvas using the GUI thread
                    skglControlCanvas.DrawPicture(m_Picture);
    
                    this.PaintCount++;
                }
            }
    
    
            // --------------------
            // --- Render Count ---
            // --------------------
    
            // Gets the number of times that this Layer has been painted
    
            public int PaintCount { get; private set; }
    
    
            // ------------------
            // --- Invalidate ---
            // ------------------
    
            // Forces the Layer to be redrawn with the next rendering cycle
    
            public void Invalidate()
            {
                m_IsValid = false;
            }
        }
    
    
        // ---------------------------------------------------------------------
        // ---------------------------------------------------------------------
        // -------                                                       -------
        // -------                    EventArgs - Draw                   -------
        // -------                                                       -------
        // ---------------------------------------------------------------------
        // ---------------------------------------------------------------------
    
    
        public class EventArgs_Draw : EventArgs
        {
            public SKRect Bounds { get; set; }
            public SKCanvas Canvas { get; set; }
    
            public EventArgs_Draw(SKCanvas canvas, SKRect bounds)
            {
                this.Canvas = canvas;
                this.Bounds = bounds;
            }
        }
    
    }