使用WebRTC流HTML5画布活动

Question

I'd like to stream the activity of an HTML5 canvas using WebRTC. One user broadcast his canvas and the connected peer see what the remote peer is doing on his canvas, if he is drawing a line or anything, i'd like the connected peer to see the changes on his own canvas.

My question, is this possible out of the box with WebRTC? After my searches, i would say that it's not the case.

If not, how would you approach such a system?

Thanks.

Answer 1

Yes there is; the CANVAS element comes with a captureStream method that allows you to capture the canvas as a RTC Peer Channel compatible MediaStream. Just as you send a camera stream to another peer you can send this captured stream to the peer to get it working.

Ref: https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream

Granted it's not supported in all browsers; but its a useful technology as it can be used for CANVAS streaming the way you've mentioned.

I'm surprised to see there's aa 2015 answer saying this isn't possible - the captureStream method on canvas has been around since 2014-2015... There's quite a few demos available on it today maybe that'll help you get started.

Answer 2

It can't be done out of the box unfortunately, as canvas cannot be used as a video source.

This has to be solved locally on each client by installing a pseudo-camera (driver emulating a camera) which records the screen instead. Then this can be used as source for WebRTC.

You can use something like WebSocket to emulate streaming by sending draw commands by the server and replay them on receiver end, but it won't be an integrated part of WebRTC of course (or use its data channel).

Update
This answer was written 1.5+ years ago and although there existed initial drafts for a captureStream() interface it was in its very infant state. The technology has matured since then and can be used for experimental usage.

However, be warned as the page also states:

This is an experimental technology

as well as

Also note that the syntax and behavior of an experimental technology is subject to change in future versions of browsers as the specification changes.

The feature is only supported in Chrome (and therefor also in the new Opera) and Firefox, and must currently be enabled by flags.

In other words, it's not something to consider for production quite yet.

Answer 3

Very few things are "out of the box" with WebRTC; What you want to do isn't the exception. Yet, it could be achieved, all depending on the complexity of your canvas application.

An idea can be to render a local canvas representing the remote canvas. Once the peers are connected, the remote peer can send the information needed for you to duplicate the contents of the remote canvas on your own.

If that is possible, then a way to send this information is over WebRTC's DataChannel, which exposes an API similar to WebSockets. It is certainly faster than WebSockets because of the nature of the peer connection, and the ability to send packets with out-of-order delivery and variable amount of re transmit.

Relevant information here: http://www.html5rocks.com/en/tutorials/webrtc/datachannels/