vp9 处理数据包丢失还是我必须手动处理？

Question

I am capturing my screen in real-time and encoding them using the vp9 codec (using JNI). encoded frames are I-frame or P-frame. then I divide them into chunks (sub-frames) and send them to the.network. But at receiving end there has been some natural packet loss and even a single miss of sub-frames causing the inability to reconstruct corresponding I/P - frames. I tried to simulate (randomly throwing out some sub-frames) the same thing locally and the same things happened. Doesn't VP9 codec has some built-in packet loss handling? If so, how to enable them and how can it perform well to a certain percentage?

And if there is no built-in packet loss handling Do I have to implement FIR or FEC manually? and where to follow? Thanks in advance.

Answer 1

Common way to send video stream is RTP protocol based on UDP, among other libs WebRTC also uses this transport under hood. Each encoded frame before sending is packetized, ie splitted to one or several RTP packets. In this context term "packet loss" means RTP packet loss. These losses are handled by sender peer using RTCP Receiver Reports from the other peer: the sender can retransmit lost packets. So, such reconstruction is not related to VP9 or any other specific codec.

Answer 2

As vp9 is a entropy coding, even a single packet missing causes inability to reconstruct the I/P frame. Even inability to reconstruct I frame cause inability to construct all subsequent & dependent P frame's. As I am using raw vp9 I have to implement any kind of retransmission or redundancy.

There is a concept of error_resilient packet or golden frame which can be called as budget version of I-frame which I need to send at certain interval from sender so that the I frame & subsequent P frame's will have some resiliency (I tried it and failed generating golden frame from encoder by enabling the parameter, maybe I will have to generate it myself).