使用 WebRTC 从画布元素流式传输视频

Question

I am using WebRTC for peer-to-peer video communication, and I would like to apply video filters to local webcam video before sending it to a remote peer.

The approach that I am considering is to send the local webcam video to a canvas element, where I will apply javascript filters to the video. Then I would like to stream the video from the canvas element to the peer using WebRTC. However, it is not clear to me if this is possible.

Is it possible to stream video from a canvas element using WebRTC? If so, how can this be done? Alternatively, are there any other approaches that I might consider to accomplish my objective?

Answer 1

It's April 2020; you can achieve this with the canvas.captureStream() method.

There is an excellent article on how to use it, along with several demos on github. See the following links:

Capture Stream

Stream from a canvas element to peer connection

So, basically, you can apply all the transformations on the canvas and stream from the canvas to remote peer.

Answer 2

mozCaptureStreamUntilEnded is supported on firefox but resulting stream can't be attached to peer connection.

Playing over <canvas> is easier however streaming media from a <video> element requires Media Processing API (capture-stream-until-ended) along with RTCPeerConnection (with all features support).

We can get images from <canvas> however I'm not sure if we can generate MediaStream from <canvas> .

So, mozCaptureStreamUntilEnded is useful only wth pre-recorded media streaming .

Answer 3

我的解决方案是，将普通流发送到对等方，也传输，它必须如何修改，所以在另一方面，而不是直接在视频元素中显示（播放视频 n 隐藏元素），你会保持使用 settimeout/requestAnimationFrame 在画布中绘制（处理后）。