简体繁体 English

如何在 WebRTC 中使用 FFmpeg H264 编码器？

[英]How to use FFmpeg H264 encoder in WebRTC?

原文 2017-07-19 15:02:21 3 2 html/ video/ ffmpeg/ webrtc/ h.264

For H264 encoding WebRTC uses OpenH264 which does not support hardware acceleration.对于 H264 编码，WebRTC 使用不支持硬件加速的 OpenH264。 There are many third party codecs included in WebRTC including WebRTC. WebRTC 中包含许多第三方编解码器，包括 WebRTC。 How FFmpeg can be used instead?如何改用 FFmpeg？ "is_component_ffmpeg=true" does not seem to do anything. “is_component_ffmpeg=true”似乎没有做任何事情。

The goal here is to encode with hardware acceleration to have reduced latency and cpu usage.这里的目标是使用硬件加速进行编码，以减少延迟和 CPU 使用率。 We have hardware encoder running but do not know how to plug that into webrtc.我们正在运行硬件编码器，但不知道如何将其插入 webrtc。 Using hardware acceleration is the closest option.使用硬件加速是最接近的选项。

Where do we need to look at to use FFmpeg?使用FFmpeg需要注意哪些地方？ or use externally encoded h264 data stream?或使用外部编码的 h264 数据流？

2 个解决方案

We ended up modifying h264_encoder_impl by replacing all OpenH264 API calls with our own encoder calls.我们最终通过用我们自己的编码器调用替换所有 OpenH264 API 调用来修改h264_encoder_impl 。

WebRTC constantly keeps asking the encoder implementation to update the bitrate and framerate as it sees fit for current available bandwidth. WebRTC 不断要求编码器实现更新比特率和帧率，因为它认为适合当前可用的带宽。 The HW encoder we used supported updating only bitrates on the fly and that worked fine with WebRTC.我们使用的硬件编码器仅支持动态更新比特率，并且与 WebRTC 配合得很好。 Framerate was set to a fixed value.帧率设置为固定值。

As we did not change framerate as per wishes of WebRTC and it still worked fine, I think that encoded stream can also be sent the same way after doing only RTPFragmentation properly for given encoded buffer.由于我们没有按照 WebRTC 的要求更改帧速率并且它仍然可以正常工作，我认为在对给定的编码缓冲区仅正确执行RTPFragmentation后，编码流也可以以相同的方式发送。

We've attempted to shunt the encoding portion of the WebRTC project in the past with little luck (we wanted to pass through data that had already been encoded to multiple WebRTC clients).我们过去曾尝试分流 WebRTC 项目的编码部分，但运气不佳（我们希望将已编码的数据传递给多个 WebRTC 客户端）。 My impression is that it's very tightly integrated with quality of service.我的印象是它与服务质量紧密结合。 WebRTC wants to adjust the encoder settings based on current network traffic. WebRTC 希望根据当前网络流量调整编码器设置。

The best solution we found is to actually roll your own WebRTC using the dtlssrtpenc , nicesink , and nicesrc elements from the OpenWebRTC project:我们找到的最佳解决方案是使用 OpenWebRTC 项目中的dtlssrtpenc 、 nicesink和nicesrc元素实际推出您自己的 WebRTC：

https://github.com/EricssonResearch/openwebrtc-gst-plugins https://github.com/EricssonResearch/openwebrtc-gst-plugins

This wasn't at all easy to do.这一点都不容易。 WebRTC has a very complicated handshake and those GStreamer elements require a lot of special hookup, but it did yield the desired results. WebRTC 有一个非常复杂的握手，那些 GStreamer 元素需要很多特殊的连接，但它确实产生了预期的结果。

Oh and btw our experience is that openh264 works quite well for WebRTC traffic and we ended up using it for many cases.哦，顺便说一句，我们的经验是 openh264 对 WebRTC 流量非常有效，我们最终在许多情况下使用它。