Possible to Combine Live m3u8 stream with PIP overlay from WebRTC source?

Question

Can someone tell me what server-side technology (perhaps ffmpeg), one could use in order to:

1) display this full-screen live-streaming video:

http://aolhdshls-lh.akamaihd.net/i/gould_1@134793/master.m3u8

2) and overlay it in the lower-right corner with a live video coming from a webRTC video-chat stream?

3) and send that combined stream into a new m3u8 live-stream

4) Note that it needs to be a server-side solution - - - cannot launch multiple video players in this case (needs to pass the resulting stream to SmartTV's which only have one video-decoder at a time)

The closest example I've found so far is this article:

https://trac.ffmpeg.org/wiki/Create%20a%20mosaic%20out%20of%20several%20input%20videos

Which isn't really live, nor is it really doing overlays.

any advice is greatly appreciated.

Answer 1

Let me clear what you want in this case :

1. Input video is HLS streaming from webRTC : What about delay? is dealy important thing in your work?

2. Overlay image into video : This will need that decoding input video, and filtering it, encoding again. so it needs a lot of cpu resource and even more if input video is 1080p.

3. Re-struct new HLS format : You must put it lot of encoding option to make sure that ts fragment works well. most important thing is GOP size and ts duration.

4. You need a web server to provide m3u8 index file. you can use nginx, apache.

What i tell you now in this answer is that ffmpeg command line, which making overlay from input HLS streaming and re-make ts segments.

Following command-line will do what you want in step 1 to step 3 :

ffmpeg \
-re -i "http://aolhdshls-lh.akamaihd.net/i/gould_1@134793/master.m3u8" \
-i "[OVERLAY_IMAGE].png" \
-filter_complex "[0:v][1:v]overlay=main_w:main_h[output]" \
-map [output] -0:a -c:v libx264 -c:a aac -strict -2 \
-f ssegment -segment_list out.list out%03d.ts

This is basic command line that overlay image from your input HLS streaming then creating ts segment and index file.

I don't have any further experience with HLS so it can be done without any tuning option but maybe you should tune it for you work. and also you should search a little bit for web server to provide m3u8 but it won't be hard.

GOP size(-g) and its duration(segment_tim), as i said, will be key point of your tuning.