Using ffprobe/ffmpeg to extract individual frames and types in encode order

Question

I am able to extract keyframes using ffmpeg. Something like this that I have been using:

ffmpeg -i input.mp4 -vf "select='eq(pict_type\,I)" -vsync vfr -qscale:v 2 I-thumbnails-%02d.png -vf "select='eq(pict_type\,B)" -vsync vfr -qscale:v 2 B-thumbnails-%02d.png -vf "select='eq(pict_type\,P)" -vsync vfr -qscale:v 2 P-thumbnails-%02d.png

Now the issue is, I would like these extracted frames to be in encode order, if possible, the way they are extracted should have a timestamp or any way to know that they start in a certain sequence, example, from start to end:

IBBBIPPB......BI

but in a way that I can sort the frames in the encode sequence.

I want to use this to load in python to extract motion vectors but they should all follow the encoding sequence. Any help?

Edit: Changed playback to encode sequence(or order).

Answer 1

With a recent version of ffmpeg, you can run,

ffmpeg -i input.mp4
  -vf "select='eq(pict_type\,I)" -vsync 0 -frame_pts 1 thumbnails-%02d-I.png
  -vf "select='eq(pict_type\,B)" -vsync 0 -frame_pts 1 thumbnails-%02d-B.png
  -vf "select='eq(pict_type\,P)" -vsync 0 -frame_pts 1 thumbnails-%02d-P.png

The image serial numbers will correspond to their index position (display-wise) in the video, and the suffix will indicate the frame type.

---

To get frames in encode/decode order, run

ffmpeg -i input.mp4
  -vf "setpts=POS,select='eq(pict_type\,I)" -vsync 0 -frame_pts 1 thumbnails-%09d-I.png
  -vf "setpts=POS,select='eq(pict_type\,B)" -vsync 0 -frame_pts 1 thumbnails-%09d-B.png
  -vf "setpts=POS,select='eq(pict_type\,P)" -vsync 0 -frame_pts 1 thumbnails-%09d-P.png

You should sort the output image listing in alphabetical/lexical order - that's the images in encode/decode order. You can batch rename the 9-digit field to a simple serial index, if wanted.

setpts=POS sets the frame's timestamp to its byte offset in the file, which will track the encode/decode order.