AVPlayer Callback (Observer) for Frame Changed

Question

Is there way to set up an observer / callback on an AVPlayer to get notified when the frame changes?

I am aware of both addBoundaryTimeObserver and addPeriodicTimeObserver however these are approximations that require me to estimate the frame rate, etc.

There is a note that:

General State Observations: You can use Key-value observing (KVO) to observe state changes to many of the player's dynamic properties, such as its currentItem or its playback rate. You should register and unregister for KVO change notifications on the main thread. This avoids the possibility of receiving a partial notification if a change is being made on another thread. AV Foundation invokes observeValue(forKeyPath:of:change:context:) on the main thread, even if the change operation is made on another thread.

However currentTime on AVPlayerItem is a method, not a property so I cannot use KVO for that.

Answer 1

You could add an AVPlayerItemVideoOutput to your AVPlayerItem and periodically poll the output with hasNewPixelBufferForItemTime which will tell you of the arrival of a new frame. However you then need to acquire the frame with copyPixelBufferForItemTime , so you should probably immediately release it. Here's an example of setting up AVPlayerItemVideoOutput . This is polling, so you could realise late or even miss a frame change.

You could also quickly preprocess the video file (if it is a file) without decompressing the frames, to determine the frame presentation time stamps. You could feed those timestamps one at a time to addBoundaryTimeObserver to decide when you'd crossed a frame boundary. Here's an example of parsing a video file .

AVSampleBufferDisplayLayer , which is a lower level AVPlayerLayer , that lets you feed it video frame CMSampleBuffer s looks like a promising way to find out when a frame changes, but it doesn't seem to tell you when it has displayed one of the sample buffers you gave it. And I don't think AVSampleBufferDisplayLayer handles audio, either.

You could also reimplement the AVPlayer playback system - then you'd be painfully (and accurately) aware of frame changes (and audio changes, and opengl/metal). Surely that kind of effort is not required here. What kind of feature are are you trying to implement exactly?