WebRTC從畫布卡紙發送流

Question

好吧，這個問題非常罕見。 我創建了Angular應用程序，以通過WebRTC在兩個Peer之間進行通信。 該體系結構是簡單的2個對等方，它們從攝像機發送視頻流並從其他對等方接收流。 簡單且有效。

現在，我想添加一些要處理的流，因此向第一個對等方添加如下所示的canvas元素：

    this.localCameraVideoStream = document.createElement('video');
    this.localCameraVideoStream.srcObject = stream;
    this.localCameraVideoStream.muted = true;
    this.localCameraVideoStream.play();

    this.canvas = document.createElement('canvas');
    this.canvas.width = 1280;
    this.canvas.height = 720;
    this.canvasStream = this.canvas.captureStream();

    this.localVideoElement.nativeElement.srcObject = this.canvasStream;
    this.localVideoElement.nativeElement.muted = true;
    this.localVideoElement.nativeElement.play();
    this.redrawStreamToCanvas();

再繪制方法將流從<video>元素繪制到畫布：

 private redrawStreamToCanvas(){
    const ctx = this.canvas.getContext('2d');
    const width = 1280;
    const height = 720;
    const combinedVideoStream = this.localCameraVideoStream;

    function drawVideo() {
      ctx.clearRect(0, 0, width, height);
      ctx.drawImage(combinedVideoStream, 0, 0, width, height);
      requestAnimationFrame(drawVideo);
    }

    requestAnimationFrame(drawVideo);
  }

只是為了消除任何疑問。 localCameraVideoStream是創建的<video>元素，用於保存攝像機流，而<canvas>可以從<video>獲取流。 canvas是<canvas> localVideoElement只是.html中的<video> ，並顯示當前的“本地”流。

問題在於，在本地預覽中，一切正常。 當我將canvasStream發送給其他對等方並將其顯示在某些<video>我只會得到一幀而已。

你知道什么發生了嗎？ 奇怪的是，在預覽中還可以，沒有這種組合，rtc連接也可以。

Answer 1

這很可能是由於Chrome的 [功能] 錯誤 ，一旦他們離開視口，它們就會暫停靜音的 <video> 。

如果確實如此，那么這與MediaStream無關，並且已經在“ 繪制畫布”操作中發生：

 var ctx = canvas.getContext('2d'); if ((videoin.buffered && !videoin.buffered.length) || videoin.paused) { videoin.onloadedmetadata = videoin.onplaying = begin; } else { begin(); } function begin() { videoin.onloadedmetadata = videoin.onplaying = null; canvas.width = videoin.videoWidth; canvas.height = videoin.videoHeight; drawToCanvas(); } function drawToCanvas() { ctx.drawImage(videoin, 0, 0); requestAnimationFrame(drawToCanvas); } 

 body { margin-bottom: 100vh; } 

 <p> Scroll down until the &lt;video&gt; element be out. </p> <video crossorigin id="videoin" src="https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm" muted autoplay></video> <canvas id="canvas"></canvas> 

因此，如果您未設置muted屬性，則可以使用：

 var ctx = canvas.getContext('2d'); if ((videoin.buffered && !videoin.buffered.length) || videoin.paused) { videoin.onloadedmetadata = videoin.onplaying = begin; } else { begin(); } function begin() { videoin.volume = 0; // does the same you'd say? videoin.onloadedmetadata = videoin.onplaying = null; canvas.width = videoin.videoWidth; canvas.height = videoin.videoHeight; drawToCanvas(); } function drawToCanvas() { ctx.drawImage(videoin, 0, 0); requestAnimationFrame(drawToCanvas); } 

 body { margin-bottom: 100vh; } 

 <p> Scroll down until the &lt;video&gt; element be out. </p> <video crossorigin id="videoin" src="https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm" autoplay></video> <canvas id="canvas"></canvas> 

甚至，即使您沒有將其附加在文檔中，它也可以正常工作（在用戶手勢之后。）。

 var ctx = canvas.getContext('2d'); var videoin = document.createElement('video'); videoin.onloadedmetadata = videoin.onplaying = begin; videoin.muted = true; // even if 'muted' videoin.src = 'https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm'; function begin() { videoin.onplay = null; canvas.width = videoin.videoWidth; canvas.height = videoin.videoHeight; drawToCanvas(); } function drawToCanvas() { ctx.drawImage(videoin, 0, 0); requestAnimationFrame(drawToCanvas); } play_btn.onclick = e => { videoin.play(); } 

 <button id="play_btn">click to start playing the video</button> <p> This contains only the canvas element </p> <canvas id="canvas"></canvas> 

現在，我不禁在這個答案中說，在調用它的captureStream方法之前，您必須已經初始化了畫布上下文（甚至可能已經在其上繪制過）。 否則，將導致Firefox中出現NS異常，並且IIRC符合規范。