How do I overlay a gizeh vector animation onto a moviepy video with transparency?

Question

I can't work out how to overlay a gizeh animation onto a video so that the vector graphics are visible but the background is transparent so the video is visible underneath the animation. I've tried lots of different ways and nothing seems to work. All I ever get is the gizeh animation completely hiding the underlying video.

This was my latest effort, just simply trying to draw a red line over the video, I've tried using the mask_color vfx method to create a mask that uses the Surface bg_color, but it doesn't have any effect.

import gizeh
from moviepy.editor import *

def make_frame(t):
    surface = gizeh.Surface(width=720, height=1280, bg_color=(0.5, 0.5, 0))
    line = gizeh.polyline(points=[(0, 1180), (720, 1180)], stroke_width=3, stroke=(1, 0, 0))
    line.draw(surface)
    return surface.get_npimage()


original_clip = VideoFileClip("test_original_video.mp4")
graphics_clip = VideoClip(make_frame, duration=original_clip.duration)
masked_graphics_clip = vfx.mask_color(graphics_clip, [0.5, 0.5, 0])

final_clip = CompositeVideoClip(
    [original_clip,
     graphics_clip],
    size=(720, 1280))

final_clip.write_videofile("test_output_video.mp4", fps=30))

How do I define and apply the mask of the animated graphics clip?

Answer 1

Zulko, the author of moviepy and gizeh very kindly helped me find a solution to this (full details here https://github.com/Zulko/moviepy/issues/898 ).

The trick is to:

• Use the same make_frame function for both the graphics and the animation.
• Return the numpy image array with the transparent=True option, which returns an opacity value for each pixel after the RGB values [so the shape of the array is (width, height, 4)]
• For the mask clip, slice the array so it only uses the opacity value [giving a shape of (width, height, 1)]
• For the graphics clip, slice the array so it only use the RGB values [giving a shape of (width, height, 3)]
• Apply the mask clip to the graphics clip

The working code looks like this:

    import gizeh
    from moviepy.editor import *

    def make_frame(t):
        surface = gizeh.Surface(width=720, height=1280)
        line = gizeh.polyline(points=[(0, 1180), (720, 1180)], stroke_width=10, stroke=(1, 0, 0))
        line.draw(surface)
        return surface.get_npimage(transparent=True)


    original_clip = VideoFileClip("test_original_video.mp4")

    graphics_clip_mask = VideoClip(lambda t: make_frame(t)[:, :, 3] / 255.0, 
                                   duration=original_clip.duration, ismask=True)
    graphics_clip = VideoClip(lambda t: make_frame(t)[:, :, :3],
                              duration=original_clip.duration).set_mask(graphics_clip_mask)

    final_clip = CompositeVideoClip(
        [original_clip,
         graphics_clip],
        size=(720, 1280)
    )

    final_clip.write_videofile("test_output_video.mp4", fps=30)