Webgl animated texture performance versus canvas drawImage performance

Question

I'm beginning a project where I would like to slice sections from a single video and redraw those sections onto different objects on the screen. I know I can do this by using drawImage to redraw the sections of the video onto a canvas element. In doing research, it appears that it would be possible to do the same thing with an animated texture in WebGL. Does anyone know if there would be any performance gains by using a WebGL animated texture to do this rather than drawImage?

Answer 1

WebGL has an edge over 2D canvas performance wise, however, since you are dealing with video then the question would be if there is any point using WebGL.

Considering that video in the NTSC world rarely exceed 30 FPS (PAL 25 fps) you will have a nice time budget slicing the video and redrawing them. Also considering that drawImage is a very fast operation and has hardware support as well (and works without), you could weight these against bothering with WebGL which does not work on all hardware (like low-end consumer and older hardware), and support for it varies (at the moment of this writing).

I would initially stick with 2D canvas in this case. If you need to do wrapping and projecting of the slices onto 3D objects then 2D canvas is not the best choice however. You could use CSS 3D transformation for some things directly on the canvas element like quadrilateral transform for example. Just use different canvas elements each representing a slice if you need different transformations.

A trick here btw. is to create an off-screen canvas which you draw the frame into first. This way you don't have to grab from the element with a running video which may be an expensive operation depending on how the browser will grab the video bitmap data.

An example:

 var ctx = canvas.getContext('2d'), sw = 32, frame = document.createElement("canvas"), // "frame buffer" fctx = frame.getContext("2d"); frame.width = 500; frame.height = 280; video.addEventListener("playing", sliceAndDice, false); function sliceAndDice() { fctx.drawImage(video, 0, 0); // video to "frame buffer" to make it more smooth // some misc slicing for(var x = 0; x < frame.width; x += sw) { var y = Math.sin(x*1.5) * sw + 20; ctx.drawImage(frame, x , 0, sw, frame.height, // source slice x * 1.1, y, sw, frame.height); // dest. slice } requestAnimationFrame(sliceAndDice); } 

 <canvas id="canvas" width=560 height=320></canvas> <video style="display:none" id="video" width="500" height="280" preload="auto" autoplay="true"> <source src="//clips.vorwaerts-gmbh.de/big_buck_bunny.mp4" type="video/mp4"> <source src="//clips.vorwaerts-gmbh.de/big_buck_bunny.webm" type="video/webm"> <source src="//clips.vorwaerts-gmbh.de/big_buck_bunny.ogv" type="video/ogg"> </video> 

An alternative demo with 320 slices to show you can push it pretty far with just 2D canvas:

 var ctx = canvas.getContext('2d'), frame = document.createElement("canvas"), // "frame buffer" fctx = frame.getContext("2d"), dlt = 0; frame.width = video.width; frame.height = video.height; video.addEventListener("playing", sliceAndDice, false); function sliceAndDice() { fctx.drawImage(video, 0, 0); // video to frame buffer to make it smoother // some misc slicing for(var x = 0, y; x < frame.width; x++) { y = Math.sin(x*32+dlt) * 3 + 10; // "random" y pos. ctx.drawImage(frame, x, 0, 1, frame.height, // source slice x, y, 1, frame.height); // dest. slice } dlt += 0.2; requestAnimationFrame(sliceAndDice); }; 

 <canvas id="canvas" width=560 height=320></canvas> <video style="display:none" id="video" width="500" height="280" preload="auto" autoplay="true"> <source src="//clips.vorwaerts-gmbh.de/big_buck_bunny.mp4" type="video/mp4"> <source src="//clips.vorwaerts-gmbh.de/big_buck_bunny.webm"type="video/webm"> <source src="//clips.vorwaerts-gmbh.de/big_buck_bunny.ogv" type="video/ogg"> </video>