libavcodec : how to encode with h264 codec ,with mp4 container using controllable frame rate and bitrate(through c code)

Question

I am trying to record the screen of a pc and encode the recorded frames using h264 encoder and wrap them into a mp4 container.I want to do this because this super user link https://superuser.com/questions/300897/what-is-a-codec-eg-divx-and-how-does-it-differ-from-a-file-format-eg-mp/300997#300997 suggests it allows good trade-off between size and quality of the output file.

The application I am working on should allow users to record a few hours of video and have the minimum output file size with decent quality.

The code I have cooked up so far allows me to record and save .mpg(container) files with the mpeg1video encoder

Running:

ffmpeg -i test.mpg

on the output file gives the following output:

 [mpegvideo @ 028c7400] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegvideo, from 'test.mpg':
  Duration: 00:00:00.29, bitrate: 104857 kb/s
    Stream #0:0: Video: mpeg1video, yuv420p(tv), 1366x768 [SAR 1:1 DAR 683:384], 104857 kb/s, 25 fps, 25 tbr, 1200k tbn, 25 tbc

I have these settings for my output:

 const char * filename="test.mpg";
    int codec_id= AV_CODEC_ID_MPEG1VIDEO;
    AVCodec *codec11;
    AVCodecContext *outContext= NULL;
    int got_output;
    FILE *f;
    AVPacket pkt;
    uint8_t endcode[] = { 0, 0, 1, 0xb7 };

    /* put sample parameters */
    outContext->bit_rate = 400000;
    /* resolution must be a multiple of two */
    outContext->width=pCodecCtx->width;
    outContext->height=pCodecCtx->height;
    /* frames per second */
    outContext->time_base.num=1;
    outContext->time_base.den=25;
    /* emit one intra frame every ten frames
     * check frame pict_type before passing frame
     * to encoder, if frame->pict_type is AV_PICTURE_TYPE_I
     * then gop_size is ignored and the output of encoder
     * will always be I frame irrespective to gop_size
     */
    outContext->gop_size = 10;
    outContext->max_b_frames = 1;
    outContext->pix_fmt = AV_PIX_FMT_YUV420P;

When I change int codec_id= AV_CODEC_ID_MPEG1VIDEO to int codec_id= AV_CODEC_ID_H264 i get a file that does not play with vlc.

I have read that writing the

uint8_t endcode[] = { 0, 0, 1, 0xb7 };

array at the end of your file when finished encoding makes your file a legitimate mpeg file.It is written like this:

 fwrite(endcode, 1, sizeof(endcode), f);
    fclose(f);

in my code. Should I do the same thing when I change my encoder to AV_CODEC_ID_H264?

I am capturing using gdi input like this:

AVDictionary* options = NULL;
    //Set some options
    //grabbing frame rate
    av_dict_set(&options,"framerate","30",0);
    AVInputFormat *ifmt=av_find_input_format("gdigrab");
    if(avformat_open_input(&pFormatCtx,"desktop",ifmt,&options)!=0){
        printf("Couldn't open input stream.\n");
        return -1;
        }

I want to be able to modify my grabbing rate to optimize for the outptut file size but When I change it to 20 for example I get a video that plays so fast.How do I get a video that plays with normal speed with frames captured at 20 fps or any lower frame rate value?

While recording I get the following output on the standard error output:

[gdigrab @ 00cdb8e0] Capturing whole desktop as 1366x768x32 at (0,0)
Input #0, gdigrab, from '(null)':
  Duration: N/A, start: 1420718663.655713, bitrate: 1006131 kb/s
    Stream #0:0: Video: bmp, bgra, 1366x768, 1006131 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
[swscaler @ 00d24120] Warning: data is not aligned! This can lead to a speedloss
[mpeg1video @ 00cdd160] AVFrame.format is not set
[mpeg1video @ 00cdd160] AVFrame.width or height is not set
[mpeg1video @ 00cdd160] AVFrame.format is not set
[mpeg1video @ 00cdd160] AVFrame.width or height is not set
[mpeg1video @ 00cdd160] AVFrame.format is not set

How do I get rid of this error in my code?

In summary: 1) How do I encode h264 video wrapped into mp4 container?

2) How do I capture at lower frame rates and still play the encoded video at normal speed?

3) How do I set the format(and which format--depends on the codec?) and width and height info on the frames I write?

The code I am using in its entirety is shown below

extern "C"
{
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libswscale/swscale.h"
#include "libavdevice/avdevice.h"


#include <libavutil/opt.h>
#include <libavutil/channel_layout.h>
#include <libavutil/common.h>
#include <libavutil/imgutils.h>
#include <libavutil/mathematics.h>
#include <libavutil/samplefmt.h>
//SDL
#include "SDL.h"
#include "SDL_thread.h"
}

//Output YUV420P
#define OUTPUT_YUV420P 0
//'1' Use Dshow
//'0' Use GDIgrab
#define USE_DSHOW 0

int main(int argc, char* argv[])
{

    //1.WE HAVE THE FORMAT CONTEXT
    //THIS IS FROM THE DESKTOP GRAB STREAM.
    AVFormatContext *pFormatCtx;
    int             i, videoindex;
    AVCodecContext  *pCodecCtx;
    AVCodec         *pCodec;

    av_register_all();
    avformat_network_init();

    //ASSIGN STH TO THE FORMAT CONTEXT.
    pFormatCtx = avformat_alloc_context();

    //Register Device
    avdevice_register_all();
    //Windows
#ifdef _WIN32
#if USE_DSHOW
    //Use dshow
    //
    //Need to Install screen-capture-recorder
    //screen-capture-recorder
    //Website: http://sourceforge.net/projects/screencapturer/
    //
    AVInputFormat *ifmt=av_find_input_format("dshow");
    //if(avformat_open_input(&pFormatCtx,"video=screen-capture-recorder",ifmt,NULL)!=0){
    if(avformat_open_input(&pFormatCtx,"video=UScreenCapture",ifmt,NULL)!=0){
        printf("Couldn't open input stream.\n");
        return -1;
    }
#else
    //Use gdigrab
    AVDictionary* options = NULL;
    //Set some options
    //grabbing frame rate
    av_dict_set(&options,"framerate","30",0);
    //The distance from the left edge of the screen or desktop
    //av_dict_set(&options,"offset_x","20",0);
    //The distance from the top edge of the screen or desktop
    //av_dict_set(&options,"offset_y","40",0);
    //Video frame size. The default is to capture the full screen
    //av_dict_set(&options,"video_size","640x480",0);
    AVInputFormat *ifmt=av_find_input_format("gdigrab");
    if(avformat_open_input(&pFormatCtx,"desktop",ifmt,&options)!=0){
        printf("Couldn't open input stream.\n");
        return -1;
    }

#endif
#endif//FOR THE WIN32 THING.

    if(avformat_find_stream_info(pFormatCtx,NULL)<0)
    {
        printf("Couldn't find stream information.\n");
        return -1;
    }
    videoindex=-1;
    for(i=0; i<pFormatCtx->nb_streams; i++)
        if(pFormatCtx->streams[i]->codec->codec_type
                ==AVMEDIA_TYPE_VIDEO)
        {
            videoindex=i;
            break;
        }
    if(videoindex==-1)
    {
        printf("Didn't find a video stream.\n");
        return -1;
    }
    pCodecCtx=pFormatCtx->streams[videoindex]->codec;
    pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
    if(pCodec==NULL)
    {
        printf("Codec not found.\n");
        return -1;
    }
    if(avcodec_open2(pCodecCtx, pCodec,NULL)<0)
    {
        printf("Could not open codec.\n");
        return -1;
    }

    //THIS IS WHERE YOU CONTROL THE FORMAT(THROUGH FRAMES).
    AVFrame *pFrame;

    pFrame=av_frame_alloc();

    int ret, got_picture;

    AVPacket *packet=(AVPacket *)av_malloc(sizeof(AVPacket));

    //TRY TO INIT THE PACKET HERE
     av_init_packet(packet);


    //Output Information-----------------------------
    printf("File Information---------------------\n");
    av_dump_format(pFormatCtx,0,NULL,0);
    printf("-------------------------------------------------\n");


//<<--FOR WRITING MPG FILES
    //<<--START:PREPARE TO WRITE YOUR MPG FILE.

    const char * filename="test.mpg";
    int codec_id= AV_CODEC_ID_MPEG1VIDEO;



    AVCodec *codec11;
    AVCodecContext *outContext= NULL;
    int got_output;
    FILE *f;
    AVPacket pkt;
    uint8_t endcode[] = { 0, 0, 1, 0xb7 };

    printf("Encode video file %s\n", filename);

    /* find the mpeg1 video encoder */
    codec11 = avcodec_find_encoder((AVCodecID)codec_id);
    if (!codec11) {
        fprintf(stderr, "Codec not found\n");
        exit(1);
    }

    outContext = avcodec_alloc_context3(codec11);
    if (!outContext) {
        fprintf(stderr, "Could not allocate video codec context\n");
        exit(1);
    }

    /* put sample parameters */
    outContext->bit_rate = 400000;
    /* resolution must be a multiple of two */

    outContext->width=pCodecCtx->width;
    outContext->height=pCodecCtx->height;


    /* frames per second */
    outContext->time_base.num=1;
    outContext->time_base.den=25;

    /* emit one intra frame every ten frames
     * check frame pict_type before passing frame
     * to encoder, if frame->pict_type is AV_PICTURE_TYPE_I
     * then gop_size is ignored and the output of encoder
     * will always be I frame irrespective to gop_size
     */
    outContext->gop_size = 10;
    outContext->max_b_frames = 1;
    outContext->pix_fmt = AV_PIX_FMT_YUV420P;

    if (codec_id == AV_CODEC_ID_H264)
        av_opt_set(outContext->priv_data, "preset", "slow", 0);

    /* open it */
    if (avcodec_open2(outContext, codec11, NULL) < 0) {
        fprintf(stderr, "Could not open codec\n");
        exit(1);
    }

    f = fopen(filename, "wb");
    if (!f) {
        fprintf(stderr, "Could not open %s\n", filename);
        exit(1);
    }


    AVFrame *outframe = av_frame_alloc();
    int nbytes = avpicture_get_size(outContext->pix_fmt,
                                   outContext->width,
                                   outContext->height);

    uint8_t* outbuffer = (uint8_t*)av_malloc(nbytes);

   //ASSOCIATE THE FRAME TO THE ALLOCATED BUFFER.
    avpicture_fill((AVPicture*)outframe, outbuffer,
                   AV_PIX_FMT_YUV420P,
                   outContext->width, outContext->height);

    SwsContext* swsCtx_ ;
    swsCtx_= sws_getContext(pCodecCtx->width,
                            pCodecCtx->height,
                            pCodecCtx->pix_fmt,
                            outContext->width, outContext->height,
                            outContext->pix_fmt,
                            SWS_BICUBIC, NULL, NULL, NULL);


    //HERE WE START PULLING PACKETS FROM THE SPECIFIED FORMAT CONTEXT.
    while(av_read_frame(pFormatCtx, packet)>=0)
    {
        if(packet->stream_index==videoindex)
        {
            ret= avcodec_decode_video2(pCodecCtx,
                                         pFrame,
                                         &got_picture,packet );
            if(ret < 0)
            {
                printf("Decode Error.\n");
                return -1;
            }
            if(got_picture)
            {

            sws_scale(swsCtx_, pFrame->data, pFrame->linesize,
                  0, pCodecCtx->height, outframe->data,
                  outframe->linesize);


            av_init_packet(&pkt);
            pkt.data = NULL;    // packet data will be allocated by the encoder
            pkt.size = 0;


            ret = avcodec_encode_video2(outContext, &pkt, outframe, &got_output);
            if (ret < 0) {
               fprintf(stderr, "Error encoding frame\n");
               exit(1);
              }

            if (got_output) {
                printf("Write frame %3d (size=%5d)\n", i, pkt.size);
                fwrite(pkt.data, 1, pkt.size, f);
                av_free_packet(&pkt);
               }

            }
        }

        av_free_packet(packet);
    }//THE LOOP TO PULL PACKETS FROM THE FORMAT CONTEXT ENDS HERE.



    //
    /* get the delayed frames */
    for (got_output = 1; got_output; i++) {
        //fflush(stdout);

        ret = avcodec_encode_video2(outContext, &pkt, NULL, &got_output);
        if (ret < 0) {
            fprintf(stderr, "Error encoding frame\n");
            exit(1);
        }

        if (got_output) {
            printf("Write frame %3d (size=%5d)\n", i, pkt.size);
            fwrite(pkt.data, 1, pkt.size, f);
            av_free_packet(&pkt);
        }
    }



    /* add sequence end code to have a real mpeg file */
    fwrite(endcode, 1, sizeof(endcode), f);
    fclose(f);

    avcodec_close(outContext);
    av_free(outContext);
    //av_freep(&frame->data[0]);
    //av_frame_free(&frame);

    //THIS WAS ADDED LATER
    av_free(outbuffer);

    avcodec_close(pCodecCtx);
    avformat_close_input(&pFormatCtx);

    return 0;
}

Thank you for your time.

Answer 1

It is possible to record and play in the normal speed.

    AVDictionary* options = NULL;
    av_dict_set( &options, "preset", "veryslow", 0 );

following presets are available :

{
"ultrafast",
"superfast",
"veryfast",
"faster",
"fast",
"medium",
"slow",
"slower",
"veryslow",
"placebo", 0
}

set the suitable preset.