[英]WebRTC iOS - H264 encode failed
I am working a video streaming app (WebRTC, iOS, Swift) - app is streaming screen recording.我正在开发一个视频流应用程序(WebRTC、iOS、Swift)——应用程序正在流式传输屏幕录制。 After a few seconds the video is frozen - encoding fails for every frame几秒钟后视频被冻结——每一帧的编码都失败
I have implemented a custom RTCVideoCapturer
based on this :我已经基于此实现了自定义RTCVideoCapturer
:
open class RTCCustomFrameCapturer: RTCVideoCapturer {
private let kNanosecondsPerSecond: Float64 = 1_000_000_000
private var nanoseconds: Float64 = 0
private let frameQueue = DispatchQueue(label: "custom.capturer.queue")
override init(delegate: RTCVideoCapturerDelegate) {
super.init(delegate: delegate)
}
public func capture(_ sampleBuffer: CMSampleBuffer) {
frameQueue.async { [weak self] in
if let self = self {
if !CMSampleBufferIsValid(sampleBuffer)
|| !CMSampleBufferDataIsReady(sampleBuffer) { return }
let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
if pixelBuffer == nil { return }
let rtcPixelBuffer = RTCCVPixelBuffer(pixelBuffer: pixelBuffer!)
let timeStampNs =
CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(sampleBuffer)) * Float64(NSEC_PER_SEC)
let videoFrame = RTCVideoFrame(
buffer: rtcPixelBuffer,
rotation: RTCVideoRotation._0,
timeStampNs: Int64(timeStampNs))
self.delegate?.capturer(self, didCapture: videoFrame)
}
}
}
}
iPhone I am testing on produces frames in size: 828x1792 .我正在测试的 iPhone 产生的帧尺寸为: 828x1792 。
WebRTC log output: WebRTC 日志 output:
(video_stream_adapter.cc:519): Removing resolution down-scaling setting.
(video_stream_adapter.cc:529): Scaling up resolution, max pixels: 2147483647
(video_stream_encoder_resource_manager.cc:505): Downgrade counts: fps: {quality:0cpu:0}, resolution {quality:0cpu:0}
(video_stream_encoder.cc:1768): Updating sink restrictions from QualityScalerResource to { }
(video_source_sink_controller.cc:68): Pushing SourceSink restrictions: max_fps=60 max_pixel_count=2147483647 target_pixel_count=null
(resource_adaptation_processor.cc:229): Resource "QualityScalerResource" signalled kUnderuse. Adapted up successfully. Unfiltered adaptations: { res=0 fps=0 }
(video_adapter.cc:275): Frame size changed: scaled 336 / out 340 / in 340 Changes: 6 Input: 828x1792 Scale: 1/1 Output: 828x1792 fps: 60/60 alignment: 2
(video_stream_encoder.cc:1133): Video frame parameters changed: dimensions=828x1792, texture=1.
(video_stream_encoder.cc:693): ReconfigureEncoder:
Simulcast streams:
0: 828x1792 fps: 60 min_kbps: 30 target_kbps: 2500 max_kbps: 2500 max_fps: 60 max_qp: 56 num_tl: 1 active: true
(RTCVideoEncoderH264.mm:378): Initial encoder frame rate setting 60 is larger than the maximal allowed frame rate 18.
(RTCVideoEncoderH264.mm:526): Encoder frame rate setting 39 is larger than the maximal allowed frame rate 18.
(connection.cc:1168): Conn[70edc00:0:Net[pdp_ip0:10.204.0.x/32:Cellular:id=5]:PmIWAMUE:1:0:local:udp:10.204.0.x:62001->hxIk2cZH:1:2122129151:local:udp:169.254.131.x:49742|C--I|-|0|0|9114475305677503998|-]: Sent STUN BINDING request, id=6b5a684d5064672f66354576, use_candidate=0, nomination=0
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(RTCVideoEncoderH264.mm:512): Failed to encode frame with code: -12902
(video_stream_encoder.cc:1423): Failed to encode frame. Error code: -1
(RTCVideoEncoderH264.mm:526): Encoder frame rate setting 39 is larger than the maximal allowed frame rate 18.
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(RTCVideoEncoderH264.mm:526): Encoder frame rate setting 39 is larger than the maximal allowed frame rate 18.
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(connection.cc:1168): Conn[6889600:0:Net[pdp_ip0:10.204.0.x/32:Cellular:id=5]:PmIWAMUE:1:0:local:udp:10.204.0.x:62001->edbds2kc:1:1686052607:stun:udp:80.208.65.x:2514|C--I|-|0|0|7241540810644798975|-]: Sent STUN BINDING request, id=73686d55355066376b566d74, use_candidate=0, nomination=0
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
(RTCVideoEncoderH264.mm:765): H264 encode failed with code: -12902
Everything seems to work fine however when I set output format to dimensions lower than actual frame before capturing the first frame:一切似乎都工作正常但是当我在捕获第一帧之前将 output 格式设置为低于实际帧的尺寸时:
if let capturer = self.videoCapturer as? RTCCustomFrameCapturer {
if !videoSourceSizeInitialized, let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
let height = CVPixelBufferGetHeight(pixelBuffer)
let width = CVPixelBufferGetWidth(pixelBuffer)
localVideoTrack?.source.adaptOutputFormat(toWidth: Int32(width - 1), height: Int32(height - 1), fps: fps)
videoSourceSizeInitialized = true
}
capturer.capture(sampleBuffer)
}
Anyone has a clue why this happens?任何人都知道为什么会发生这种情况?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.