Does stream seek order matter for ffmpeg av_seek_frame()?

Question

I am attempting to seek both audio and video streams for an mp4 using the ffmpeg av_seek_frame method.

I have encountered an issue when seeking that I have remedied by changing my seek order, but would like to make sure my fix is actually a fix and not some coincidental hack that works.

I am attempting to seek both the audio and video stream to the first packet. For video, the first packet has a pts of 0. For audio, the first packet has a pts of -1024. The video stream has an index of 0 and the audio stream has an index of 1. This has all been verified using ffprobe on the media file to view the packets and streams.

The following code does not work, it seeks both the audio and video streams to packets with pts of 0:

for (int i = format_context->nb_streams - 1; i >= 0; --i) {
    AVStream* stream = format_context->streams[i];
    av_seek_frame(format_context, i, stream->first_dts, flags);
}

But this properly seeks the video stream to pts 0 and audio stream to pts -1024:

for (int i = 0; i < format_context->nb_streams; ++i) {
    AVStream* stream = format_context->streams[i];
    av_seek_frame(format_context, i, stream->first_dts, flags);
}

Note that in the first example, audio is seeked before video, and in the second example video is seeked before audio.

Does the order of the av_seek_frame calls actually matter, or is there a bug somewhere else in my code that this just so happens to cover up?

Answer 1

Seeking affects all streams. The seek request is processed by the demuxer. It will move to a some point in the file or multiplexed stream. Subsequent demuxing will give you the packets from that point on. Separately seeking in the dexmed streams is not supported.

For your use case, you could seek to a point where demuxing will give you all the packets you require, but then skip those that are too early.

So yes, the order in your sample code above does matter. But it matters only because of the way the audio and video packets happen to be stored in your mp4 file.