iOS 录屏问题

Question

I've got some trouble with screen recording using ReplayKit

So, the strangeness begins in the startCapture 's handler closure

Every time it starts to capture and sampleType is .video , the self.videoWriter?.startSession is called, and instantaneously videoWriter.status becomes failure (3)

This behaviour is not throwing any exceptions until I finish recording:

Optional("Cannot Encode")

Optional("Optional(Error Domain=AVFoundationErrorDomain Code=-11834 "Cannot Encode" UserInfo={AVErrorMediaSubTypeKey=(\n 778924083\n), NSLocalizedFailureReason=The encoder required for this media cannot be found., AVErrorMediaTypeKey=soun, NSLocalizedDescription=Cannot Encode })") Optional("<AVAssetWriter: 0x283ef0920, outputURL = file:///var/mobile/Containers/Data/Application/A85826BA-DCBE-428B-AB9E-84D77F234FF6/Documents/Replays/capture92949FA1-F65D-48A0-8140-207BC892C204.mp4, outputFileType = public.mpeg-4>")

Error Domain=PHPhotosErrorDomain Code=3302 "(null)"

Here is my code:

let settings = [
    AVFormatIDKey: Int(AudioFormat.kAudioFormatMPEGLayer3.rawValue),
    AVNumberOfChannelsKey: 2,
    AVSampleRateKey: 44100,
    AVEncoderBitRateKey: 128000
]

let audioInput = AVAssetWriterInput(
    mediaType: .audio,
    outputSettings: settings
)

audioInput.expectsMediaDataInRealTime = true
videoWriter.add(audioInput)

self.videoWriterInput = audioInput
self.videoWriter = videoWriter

let videoSettings: [String: Any] = [
   AVVideoCodecKey: AVVideoCodecType.h264,
   AVVideoWidthKey: passingSize.width,
   AVVideoHeightKey: passingSize.height
]

self.videoWriterInput = AVAssetWriterInput(
    mediaType: AVMediaType.video,
    outputSettings: videoSettings
)

self.appAudioWriterInput = AVAssetWriterInput(
    mediaType: .audio,
    outputSettings: settings
)
self.videoWriter.add(appAudioWriterInput!)


self.recorder.startCapture(
    handler: { (sampleBuffer, sampleType, passedError) in 
       switch sampleType {
       case .video:
           self.handleSampleBuffer(sampleBuffer: sampleBuffer)
       case .audioApp:
           self.add(sample: sampleBuffer, to: appAudioWriterInput)
       default:
           break
       }
   },
   completionHandler: { error in
       if let error = error {
          print(">>> Error!")
          errorHandler(error: error)
       }
    }
)

private func handleSampleBuffer(sampleBuffer: CMSampleBuffer) {
    if videoWriter?.status == AVAssetWriter.Status.unknown {
        serialQueue.async {
            if !self.isRecording {
                self.videoWriter?.startWriting()
                self.videoWriter?.startSession(
                    atSourceTime: .zero
                )
                self.isRecording = true
            }
        }
    } else if videoWriter?.status == AVAssetWriter.Status.writing &&
        videoWriterInput?.isReadyForMoreMediaData == true {
            serialQueue.async {
                self.videoWriterInput?.append(sampleBuffer)
            }
    }
}

I'm not familiar with iOS development and I think that I'm missing something obvious.

Things I've tried:

1. Clean XCode caches
2. Run app on both IPhone and Simulator
3. Check PHPhotoLibrary.authorizationStatus() and AVCaptureDevice.authorizationStatus before recording
4. Experiment with audio and video recording settings

Here is my modifications to Info.plist

<key>NSPhotoLibraryAddUsageDescription</key>
<string>The app requires access to Photos to save media to it.</string>
<key>NSPhotoLibraryUsageDescription</key>
<string>The app requires access to Photos to interact with it.</string>
<key>PHPhotoLibraryPreventAutomaticLimitedAccessAlert</key>
<true/>

Answer 1

The solution was pretty simple: I just passed wrong variable to self.videoWriterInput

Proper initialization should be something like this:

let screenSize: CGRect = UIScreen.main.bounds
        
let videoSettings: [String: Any] = [
    AVVideoCodecKey: AVVideoCodecType.h264,
    AVVideoWidthKey: screenSize.width,
    AVVideoHeightKey: screenSize.height,
]

let newVideoWriterInput = AVAssetWriterInput(
    mediaType: AVMediaType.video,
    outputSettings: videoSettings
)

        
newVideoWriterInput.expectsMediaDataInRealTime = true
self.videoWriter?.add(newVideoWriterInput)
self.videoWriterInput = newVideoWriterInput // THE ERROR WAS HERE
        
let settings: [String : Any] = [
    AVNumberOfChannelsKey : 2,
    AVFormatIDKey : kAudioFormatMPEG4AAC,
    AVSampleRateKey: 44100,
    AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue
]

let audioInput = AVAssetWriterInput(
    mediaType: .audio,
    outputSettings: settings
)

audioInput.expectsMediaDataInRealTime = true
self.videoWriter?.add(audioInput)
self.appAudioWriterInput = audioInput