Real time video

Question

There is a lot of information scattered over the internet, but it is hard to find something concrete

The question is what is the best modern way to create a live, low latency solution for video streaming

What solutions are being used for products like Wyze cams, Ring video doorbells, etc.

What are the best solutions for Android/ios clients and for web clients

Answer 1

When you build a WebRTC conference, you can choose between different architectures:

Mesh based architectures are simple, but scale badly, since every user must send its stream to n-1 other clients. So you send the same stream multiple times. In my experience, it works well for up to 5 users, but it may be to much for bad internet connections or smartphones. For larger converences, a media server is adviced.

If you want to broadcast something (1 sender, many receivers or many senders, many receivers), you may want to use a Single Forwarding Unit (SFU). This media server receives the stream of a peer connection and broadcasts it to everyone else. So you send your outgoing stream once and it gets distributed, while you may receive the n-1 other streams. It may also transcribe your video stream (change it to supported video codecs or scale it down). Common SFUs are jitsi and there was also a node-sfu module, if I remember it correctly.

For really big converences, you may just want to use a Multiconferencing Unit (MCU). It mixes the video and audio tracks. So for 1 outgoing connection, your stream, you receive 1 stream that mixed everything from everyone else and you together. This has the least amount of network and CPU load for the peers, since you only have 1 sending and receiving connection and you do not need to display multiple videos or mix them at the client. The de facto standard media server is Kurento.

On the other hand, as soon as you remove load from the client, you put it on your media server. The MCU has to decode, mix, and re-encode the video streams, the sfu has to do less (since mostly forwarding) and the full mesh approach has no media server, so it may be the cheapest one.

Answer 2

The "low latency" requirement, and a "security cam to a few viewers" requirement really narrows down your options to WebRTC Media Server.

Check out Kurento, Unreal Media Server, Ant Media Server.

• Typically, the camera will stream video to the Media Server via RTSP. This is especially good since Opus is supported by RTSP/RTP; some IP cameras already support Opus audio (Axis brands).
• Note that some media servers will transcode your video in order to stream to WebRTC viewers in web browsers, and this process takes a lot of CPU, thus, doesn't scale well. Try to choose a media server that uses original camera-encoded content, instead of transcoding.