How to dynamically change the playback rate of video in iOS?

Question

The perfect example of what I am trying to achieve can be seen in the Flow ● Slow and Fast Motion app .

One can change the playback rate of the video by dragging points on the curve up or down. The video can also be saved in this state.

I am looking for a way to dynamically speed up/down a video , so that the playback rate can be changed while the video is being played.

Video explanation

WHAT I'VE TRIED

• The playback rate property of AVPlayer .But it Only works with a few values for playback Rate(0.50, 0.67, 0.80, 1.0, 1.25, 1.50, and 2.0 ) and one cannot save the video

• The scaleTimeRange(..) property of AVMutableComposition . But it doesn't work when you want to ramp the video for gradually decreasing slow/fast motion.

• Display video frames on screen using CAEAGLLayer and CADisplayLink . But my many attempts on trying to achieve Slow/Fast motion with this have been unsuccessful .

All this has taken me months and I'm starting to doubt if I'll be able to accomplish this at all.

Thus any suggestion , would be immensely valuable.

Answer 1

In IOS, the MPNowPlayingInfoCenter object contains a 'nowPlayingInfo' dictionary whose contents describe the item being played. It is advised that you start the playback at the 'currentplaybackrate' and then set the speed. See this thread on the developer's forum.

You might possibly end up with something like this (but this is javascript) where the playback rate of the video has been sped up by 4.

  document.querySelector('video').playbackRate = 4.0; document.querySelector('video').play(); 

 video{width:400px; height:auto;} 

 <video controls preload="true" autoplay> <source src="http://www.rachelgallen.com/nature.mp4" type="video/mp4" > </video> 

Answer 2

So I'm not sure I fully understand the use case you're going for, but I think

func setRate(_ rate: Float, 
      time itemTime: CMTime, 
      atHostTime hostClockTime: CMTime) 

[Apple Documentation Source]

Is something that you're looking for. While this may not be exactly what you need, I'm also not sure where in the docs there is exactly what you're looking for, but with the above method alone, you could do the following to save videos at a variable rate:

1) Use the above method to play the video throughout (assuming it's not too long, otherwise this will be computationally impossible/timeout-worthy on some devices) at the desired rates each second. Design UI to adjust this per second rate.

2) under the hood you can actually play the video at that speed "frame by frame" and capture the frames you want (in the right # which will give you the rate you desire) and voila -- saving the right number of frames together (skipping/duplicating as needed to increase/lower desired rate based on "picker" UI) you've now accomplished what you desire

To be clear, what I'm talking about here is a video output @ 60FPS has 60 frames per second. You would literally "cut and paste" frames together from the source video into the "destination" video based on whatever UI steppers values you receive from your user (in the screenshot-ed example the question contains, as my basis), and pick up that many frames. AKA if the user says seconds 2-10 of their 20 second video should be at 2X, only put in 30 frames for each of those seconds (if filmed at 60 FPS) alternating frames. The output will, at 60FPS, seem like 2X speed (since there are now 30 frames per 1 second of original video, which is 0.5 seconds at 60 FPS). Similarly, any value can appropriately be factored into:

• (desired consistent FPS) = (source video FPS) = (destination video FPS) (ie 60 or 90)
• (rate) = (rate from UI steppers/graph UI to pick rate @ each time interval) (ie 1X/2x/0.25X)
• (desired consistent FPS) * (rate) = (# frames kept in destination video)
• (destination video frames) = (source video) * (desired consistent FPS) ~modulated by~ (per custom time interval rate)

The exact mechanisms for ^^ might actually be built into AVPlayer and I didn't find the details, but this alone should be a good start to get you going in that direction.