Create video from UI controls in UWP using Composition

Question

I'm creating an application that takes a video and overlays sensor data from a drone.

Please, watch it in action here:

https://youtu.be/eAOjImJci3M

But that doesn't edit the video, it's just on the application.

My goal is to produce a new video with the overlaid data on it. How can I capture an element of the UI over time and make a MediaClip out of it?

Answer 1

Short answer : Take a screenshot of the UI with a certain interval, and then accumulate them to produce a video:

Long answer : Let's name the UI you want to record as myGrid , to get screenshots in an interval you can use a DispatcerTimer , and handle the Tick event like this:

private async void Tm_Tick(object sender, object e)
{
    RenderTargetBitmap rendertargetBitmap = new RenderTargetBitmap();
    await rendertargetBitmap.RenderAsync(myGrid);
    var bfr = await rendertargetBitmap.GetPixelsAsync();

    CanvasRenderTarget rendertarget = null
    using (CanvasBitmap canvas = CanvasBitmap.CreateFromBytes(CanvasDevice.GetSharedDevice(), bfr, rendertargetBitmap.PixelWidth, rendertargetBitmap.PixelHeight, Windows.Graphics.DirectX.DirectXPixelFormat.B8G8R8A8UIntNormalized))
    {
        rendertarget = new CanvasRenderTarget(CanvasDevice.GetSharedDevice(), canvas.SizeInPixels.Width, canvas.SizeInPixels.Height, 96);
        using (CanvasDrawingSession ds = rendertarget.CreateDrawingSession())
        {
            ds.Clear(Colors.Black);
            ds.DrawImage(canvas);
        }
    }
    MediaClip m = MediaClip.CreateFromSurface(rendertarget, TimeSpan.FromMilliseconds(80));
    mc.Clips.Add(m);
}

mc is the MediaComposition object I declared earlier .

When you are done recording, stop the DispatcherTimer and save the video on the disk like this:

tm.Stop();
await mc.RenderToFileAsync(file, MediaTrimmingPreference.Precise, MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Vga));

tm is the DispatcherTimer I declared earlier and file is a StorageFile with mp4 extension.

This procedure doesn't require you to save each screenshot to disk .

If you wonder why I used an additional CanvasRenderTarget , it's because of this problem.

Hope that helps.