使用 canvas.captureStream() 使用 alpha 通道捕获视频

Question

I'm trying to capture the contents of a canvas element with alpha channel. When I do it I get the RGB values correctly but the Alpha channel seems to get dropped when playing back the resulting video. Is there a way to achieve this?

I'm running the following code:

var stream = canvas.captureStream(60);
var recorder = new MediaRecorder(stream);

recorder.addEventListener('dataavailable', finishCapturing);
recorder.addEventListener('stop', function(e) {
    video.oncanplay = video.play;
    video.src = URL.createObjectURL(new Blob(blobs, {type:"video/webm; codecs=vp9"}));
});
startCapturing();
recorder.start();

Here's a plnkr demonstrating the issue: http://plnkr.co/edit/z3UL9ikmn6PtVoAHvY0B?p=preview

Answer 1

There is currently no options to enable (VP8/9 transparency channel) from the MediaRecorder API.
One could maybe open an issue on the W3C Mediacapture-Record git repo.

For this, I can guess a few reasons :

• From what I understand, webm alpha channel is grossly an hack from chrome, and is not really implemented in the codec itself, nor completely stabilized.

• MediaRecorder should be able to encode in many formats even if current implementations only support video webm/VP8 and webm/VP9 (chrome only). So it would mean that they would have to somehow keep the alpha channel in the raw stream, only for this new canvas.captureStream method. Historically, MediaStream mainly came from getUserMedia interface, and there were no need way of getting transparency from there.
  [edit: Specs have changed since this answer was written, and MediaStreams should now keep the alpha channel, even if the consumer may not be able to use it, also Chrome now supports more video codecs.]

• Chrome, which is the only one to support YUVA webm display in its stable channel (FF supports it in nightly 54), is still not able to include the duration inside their recorded files, let's them fix this before adding the hackish alpha_mode=true .

However, you can achieve it yourself kind of easily :

If you really want a transparent webm file (only readable in chrome and FF nightly), you can use a second canvas to do the recording, set its background (using fillRect ) to a chroma that won't appear elsewhere in your drawings, draw the original one on it and record its stream. Once recorded, use ffmpeg to reencode the recorded video, this time with the alpha channel :

// all #00FF00 pixels will become transparent.
ffmpeg -i in.webm -c:v libvpx -vf "chromakey=0x00ff00:0.1:0.1,format=yuva420p" -auto-alt-ref 0 out.webm

^{I personally needed the -auto-alt-ref 0 flag, not sure everyone needs it though}

But because of this other chrome bug , you'll actually have to append this other canvas on screen too, and hide it with css ( opacity: 0; width:0px; height:0px; should do).

TL;DR

This API's implementations are far from being stabilized, no one has made the request for such a feature yet, it may come in near future though, and can be done server-side for the time being.