[英]WebRTC sending stream from canvas jams
Well, the problem is very unusual. 好吧,这个问题非常罕见。 I create Angular app to communicate between two Peers via WebRTC. 我创建了Angular应用程序,以通过WebRTC在两个Peer之间进行通信。 The architecture was simple 2 peers which sends video stream from camera and receives stream from other peer. 该体系结构是简单的2个对等方,它们从摄像机发送视频流并从其他对等方接收流。 Simple and working. 简单且有效。
Now I want to add some processing to stream so to first peer I add the canvas element like this: 现在,我想添加一些要处理的流,因此向第一个对等方添加如下所示的canvas元素:
this.localCameraVideoStream = document.createElement('video');
this.localCameraVideoStream.srcObject = stream;
this.localCameraVideoStream.muted = true;
this.localCameraVideoStream.play();
this.canvas = document.createElement('canvas');
this.canvas.width = 1280;
this.canvas.height = 720;
this.canvasStream = this.canvas.captureStream();
this.localVideoElement.nativeElement.srcObject = this.canvasStream;
this.localVideoElement.nativeElement.muted = true;
this.localVideoElement.nativeElement.play();
this.redrawStreamToCanvas();
And redraw method to draw stream from <video>
element to canvas: 再绘制方法将流从<video>
元素绘制到画布:
private redrawStreamToCanvas(){
const ctx = this.canvas.getContext('2d');
const width = 1280;
const height = 720;
const combinedVideoStream = this.localCameraVideoStream;
function drawVideo() {
ctx.clearRect(0, 0, width, height);
ctx.drawImage(combinedVideoStream, 0, 0, width, height);
requestAnimationFrame(drawVideo);
}
requestAnimationFrame(drawVideo);
}
Just to clear any doubts. 只是为了消除任何疑问。 localCameraVideoStream is <video>
element created to save camera stream in a way that <canvas>
can get stream from <video>
. localCameraVideoStream是创建的<video>
元素,用于保存摄像机流,而<canvas>
可以从<video>
获取流。 canvas is <canvas>
localVideoElement is just a <video>
in .html and displays current "local" stream. canvas是<canvas>
localVideoElement只是.html中的<video>
,并显示当前的“本地”流。
Problem is that in local preview everything works properly. 问题在于,在本地预览中,一切正常。 When I send canvasStream
to other peer and display it on some <video>
I only get one frame and thats it. 当我将canvasStream
发送给其他对等方并将其显示在某些<video>
我只会得到一帧而已。
Do you know whats hapenning? 你知道什么发生了吗? It's weird that in preview is ok and without this combinations rtc connection is also ok. 奇怪的是,在预览中还可以,没有这种组合,rtc连接也可以。
This is most probably because of a bug [feature] of Chrome , that will pause muted <video>
once they get out of the viewport. 这很可能是由于Chrome的 [功能] 错误 ,一旦他们离开视口,它们就会暂停静音的 <video>
。
If it is indeed the case, this has nothing to do with the MediaStream, and already just happens in the drawing-to-canvas operation: 如果确实如此,那么这与MediaStream无关,并且已经在“ 绘制画布”操作中发生:
var ctx = canvas.getContext('2d'); if ((videoin.buffered && !videoin.buffered.length) || videoin.paused) { videoin.onloadedmetadata = videoin.onplaying = begin; } else { begin(); } function begin() { videoin.onloadedmetadata = videoin.onplaying = null; canvas.width = videoin.videoWidth; canvas.height = videoin.videoHeight; drawToCanvas(); } function drawToCanvas() { ctx.drawImage(videoin, 0, 0); requestAnimationFrame(drawToCanvas); }
body { margin-bottom: 100vh; }
<p> Scroll down until the <video> element be out. </p> <video crossorigin id="videoin" src="https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm" muted autoplay></video> <canvas id="canvas"></canvas>
So if you didn't set the muted
property, it would have worked: 因此,如果您未设置muted
属性,则可以使用:
var ctx = canvas.getContext('2d'); if ((videoin.buffered && !videoin.buffered.length) || videoin.paused) { videoin.onloadedmetadata = videoin.onplaying = begin; } else { begin(); } function begin() { videoin.volume = 0; // does the same you'd say? videoin.onloadedmetadata = videoin.onplaying = null; canvas.width = videoin.videoWidth; canvas.height = videoin.videoHeight; drawToCanvas(); } function drawToCanvas() { ctx.drawImage(videoin, 0, 0); requestAnimationFrame(drawToCanvas); }
body { margin-bottom: 100vh; }
<p> Scroll down until the <video> element be out. </p> <video crossorigin id="videoin" src="https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm" autoplay></video> <canvas id="canvas"></canvas>
Or even, if you didn't appended it in the document, it would also have worked (after an user-gesture..). 甚至,即使您没有将其附加在文档中,它也可以正常工作(在用户手势之后。)。
var ctx = canvas.getContext('2d'); var videoin = document.createElement('video'); videoin.onloadedmetadata = videoin.onplaying = begin; videoin.muted = true; // even if 'muted' videoin.src = 'https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm'; function begin() { videoin.onplay = null; canvas.width = videoin.videoWidth; canvas.height = videoin.videoHeight; drawToCanvas(); } function drawToCanvas() { ctx.drawImage(videoin, 0, 0); requestAnimationFrame(drawToCanvas); } play_btn.onclick = e => { videoin.play(); }
<button id="play_btn">click to start playing the video</button> <p> This contains only the canvas element </p> <canvas id="canvas"></canvas>
Now, I can't refrain from saying in this answer that you must have already initialized your canvas context (and even probably have already drawn on it) before calling its captureStream
method. 现在,我不禁在这个答案中说,在调用它的captureStream
方法之前,您必须已经初始化了画布上下文(甚至可能已经在其上绘制过)。 Failing to do this will result in an NS Exception in Firefox, and IIRC it is in accordance with the specs. 否则,将导致Firefox中出现NS异常,并且IIRC符合规范。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.