ffmpeg Get time of frames from trimmed video

Question

I am using FFmpeg in my application to extract frames from a video, the frames will be added to a trim video view where you get an illustration as to what is happening in the video at a specific time within the video. So each frame needs to represent some time within the video.

I dont quite understand how FFmpeg is producing the frames. Here is my code:

"-i",
videoCroppedFile.getAbsolutePath(),
"-vf",
"fps=1/" + frameSeperation,
mediaStorageDir.getAbsolutePath() +
"/%d.jpg"

My app allows you to record a video at a max length of 20s. The number of frames extracted from the video depnds on how long the captured video is. frameSeperation is calculated doing the below code.

String time = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
    long videoLength = Long.parseLong(time) / 1000;
    double frameSeperationDouble = (double) videoLength;
    // Divide by 11 because there is a maximum of 11 frames on trim video view
    frameSeperationDouble /= 11;
    frameSeperationDouble = Math.ceil(frameSeperationDouble);
    int frameSeperation = (int) frameSeperationDouble;

Maybe the above logic is very bad, if there is a better way please can somebody tell me.

Anyway I run the code and below are a few test cases:

• A video captured with a length of 6 seconds has 7 frames.
• A video captured with a length of 2 seconds has 3 frames.
• A video captured with a length of 10 seconds has 12 frames.
• A video captured with a length of 15 seconds has 9 frames.
• A video captured with a length of 20 seconds has 11 frames.

There is no consistency, and I find it hard to put timestamps against each frame because of this. I feel like my logic is wrong or im not understanding. Any help is much appreciated

Update 1

So I did what you said in comments:

 final FFmpeg ffmpeg = FFmpeg.getInstance(mContext);
        final File mediaStorageDir = new File(Environment.getExternalStorageDirectory()
                + "/Android/data/"
                + mContext.getPackageName()
                + "/vFrames");

    if (!mediaStorageDir.exists()){
        mediaStorageDir.mkdirs();
    }

    MediaMetadataRetriever retriever = new MediaMetadataRetriever();
    retriever.setDataSource(mContext, Uri.fromFile(videoCroppedFile));
    String time = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
    long videoLength = Long.parseLong(time) / 1000;
    double frameSeperationDouble = (double) videoLength / 8;

    retriever.release();

    final String cmd[] = {

            "-i",
            videoCroppedFile.getAbsolutePath(),
            "-vf",
            "fps=1/" + frameSeperationDouble,
            "-vframes," + 8,
            mediaStorageDir.getAbsolutePath() +
            "/%d.jpg"
    };

I also tried "-vframes=" + 8 at the same point where I put vFrames in cmd. It doesnt seem to work at all now no frames are being extracted from the video

Answer 1

This is the effective flow for fps=x , with the default rounding method near ,

Generate intervals of 1/x seconds, e,g. for x=1/3 , intervals are 0-3s, 3-6s, 6-9s... Within each interval, pick an earlier frame whose timestamp is nearest to the midpoint of that interval. This may lead to duplicate frames, when that nearest frame belongs to an earlier interval, and was already picked for that interval. Likely to happen if source is variable frame-rate.

In your case, I would suggest to avoid rounding or stripping fractional values, if possible. Or do it at the end.