Error using FFMPEG to convert each input image into H264 compiling in Visual Studio running in MevisLab

Question

I am creating a ML Module in MevisLab Framework, I am using FFMPEG to convert each image i get into a H264 Video and save it after I get all the frames. But unfortunately I have problem allocating the output buffer size. The application crashes when I include this in my code.If I am not including it, the output file size is just 4kb. Nothing is stored in it.

I am also not very sure whether it is correct way of getting the HBitmap into the Encoder. Would be great to have your suggestions.

My Code:

BITMAPINFO bitmapInfo;
            HDC        hdc;

            ZeroMemory(&bitmapInfo, sizeof(bitmapInfo));

            BITMAPINFOHEADER &bitmapInfoHeader = bitmapInfo.bmiHeader;
            bitmapInfoHeader.biSize            = sizeof(bitmapInfoHeader);
            bitmapInfoHeader.biWidth           = _imgWidth;
            bitmapInfoHeader.biHeight          = _imgHeight;
            bitmapInfoHeader.biPlanes          =  1;
            bitmapInfoHeader.biBitCount        = 24;
            bitmapInfoHeader.biCompression     = BI_RGB;
            bitmapInfoHeader.biSizeImage       = ((bitmapInfoHeader.biWidth * bitmapInfoHeader.biBitCount / 8 + 3) & 0xFFFFFFFC) * bitmapInfoHeader.biHeight;
            bitmapInfoHeader.biXPelsPerMeter   = 10000;
            bitmapInfoHeader.biYPelsPerMeter   = 10000;
            bitmapInfoHeader.biClrUsed         = 0;
            bitmapInfoHeader.biClrImportant    = 0;
            //RGBQUAD* Ref = new RGBQUAD[_imgWidth,_imgHeight];
            HDC hdcscreen = GetDC(0);

            hdc = CreateCompatibleDC(hdcscreen);
            ReleaseDC(0, hdcscreen);

            _hbitmap = CreateDIBSection(hdc, (BITMAPINFO*) &bitmapInfoHeader, DIB_RGB_COLORS, &_bits, NULL, NULL);

To get the BitMap I use the above code. Then I allocate the Codec Context as followed

c->bit_rate = 400000;
                // resolution must be a multiple of two 
                c->width = 1920;
                c->height = 1080;
                // frames per second 
                frame_rate = _framesPerSecondFld->getIntValue();
                //AVRational rational = {1,10};
                //c->time_base = (AVRational){1,25};
                 //c->time_base = (AVRational){1,25};
                 c->gop_size = 10; // emit one intra frame every ten frames 
                 c->max_b_frames = 1;
                 c->keyint_min = 1;   //minimum GOP size
                 c->time_base.num = 1;                                  // framerate numerator
                 c->time_base.den = _framesPerSecondFld->getIntValue(); 
                 c->i_quant_factor = (float)0.71;                        // qscale factor between P and I frames
                 c->pix_fmt = AV_PIX_FMT_RGB32;
                 std::string msg;
                 msg.append("Context is stored");
                 _messageFld->setStringValue(msg.c_str());

I create the Bitmap Image as followed from the input

PagedImage *inImg = getUpdatedInputImage(0);
        ML_CHECK(inImg);
        ImageVector imgExt = inImg->getImageExtent();
        if ((imgExt.x = _imgWidth) && (imgExt.y == _imgHeight))
        {
        if (((imgExt.x % 4)==0) && ((imgExt.y % 4) == 0))
        {
                 // read out input image and write output image into video
                // get input image as an array
                void* imgData = NULL;
                SubImageBox imageBox(imgExt); // get the whole image
                getTile(inImg, imageBox, MLuint8Type, &imgData);
                iData = (MLuint8*)imgData;
                int r = 0; int g = 0;int  b = 0;
                // since we have only images with
                // a z-ext of 1, we can compute the c stride as follows
                int cStride = _imgWidth * _imgHeight;
                uint8_t offset  = 0;
                // pointer into the bitmap that is
                // used to write images into the avi
                UCHAR* dst = (UCHAR*)_bits;
                for (int y = _imgHeight-1; y >= 0; y--)
                { // reversely scan the image. if y-rows of DIB are set in normal order, no compression will be available.
                    offset = _imgWidth * y;
                    for (int x = 0; x < _imgWidth; x++)
                    {
                        if (_isGreyValueImage)
                        {
                            r = iData[offset + x];
                            *dst++ = (UCHAR)r;
                            *dst++ = (UCHAR)r;
                            *dst++ = (UCHAR)r;
                        } 
                        else
                        {
                            b = iData[offset + x]; // windows bitmap need reverse order: bgr instead of rgb
                            g = iData[offset + x + cStride          ];
                            r = iData[offset + x + cStride + cStride];

                            *dst++ = (UCHAR)r;
                            *dst++ = (UCHAR)g;
                            *dst++ = (UCHAR)b;
                        }
                        // alpha channel in input image is ignored
                    }
                }

Then I add it to the Encoder as followed as write as H264

 in_width   = c->width;
                 in_height  = c->height;
                 out_width  = c->width;
                 out_height = c->height;
                 ibytes = avpicture_get_size(PIX_FMT_BGR32, in_width, in_height);
                 obytes = avpicture_get_size(PIX_FMT_YUV420P, out_width, out_height);
                 outbuf_size = 100000 + c->width*c->height*(32>>3);      // allocate output buffer
                 outbuf = static_cast<uint8_t *>(malloc(outbuf_size));

                 if(!obytes)
                 {
                     std::string msg;
                     msg.append("Bytes cannot be allocated");
                     _messageFld->setStringValue(msg.c_str());
                 }
                 else
                 {
                     std::string msg;
                     msg.append("Bytes allocation done");
                     _messageFld->setStringValue(msg.c_str());
                 }
                 //create buffer for the output image
                 inbuffer  =  (uint8_t*)av_malloc(ibytes);
                 outbuffer =  (uint8_t*)av_malloc(obytes);
                 inbuffer  =  (uint8_t*)dst;

                 //create ffmpeg frame structures.  These do not allocate space for image data, 
                 //just the pointers and other information about the image.
                 AVFrame* inpic = avcodec_alloc_frame();
                 AVFrame* outpic = avcodec_alloc_frame();

                 //this will set the pointers in the frame structures to the right points in 
                 //the input and output buffers.
                 avpicture_fill((AVPicture*)inpic, inbuffer, PIX_FMT_BGR32, in_width, in_height);
                 avpicture_fill((AVPicture*)outpic, outbuffer, PIX_FMT_YUV420P, out_width, out_height);
                 av_image_alloc(outpic->data, outpic->linesize, c->width, c->height, c->pix_fmt, 1); 
                 inpic->data[0] += inpic->linesize[0]*(_imgHeight-1);                                                      // flipping frame
                 inpic->linesize[0] = -inpic->linesize[0];    

                 if(!inpic)
                 {
                     std::string msg;
                     msg.append("Image is empty");
                     _messageFld->setStringValue(msg.c_str());
                 }
                 else
                 {
                     std::string msg;
                     msg.append("Picture has allocations");
                     _messageFld->setStringValue(msg.c_str());
                 }

                 //create the conversion context
                 fooContext = sws_getContext(in_width, in_height, PIX_FMT_BGR32, out_width, out_height, PIX_FMT_YUV420P, SWS_FAST_BILINEAR, NULL, NULL, NULL);
                 //perform the conversion
                 sws_scale(fooContext, inpic->data, inpic->linesize, 0, in_height, outpic->data, outpic->linesize);
                 //out_size = avcodec_encode_video(c, outbuf,outbuf_size, outpic);
                 if(!out_size)
                 {
                     std::string msg;
                     msg.append("Outsize is not valid");
                     _messageFld->setStringValue(msg.c_str());
                 } 
                 else
                 {
                     std::string msg;
                     msg.append("Outsize is valid");
                     _messageFld->setStringValue(msg.c_str());
                 }
                     fwrite(outbuf, 1, out_size, f);
                     if(!fwrite)
                 {
                     std::string msg;
                     msg.append("Frames couldnt be written");
                     _messageFld->setStringValue(msg.c_str());
                 } 
                 else
                 {
                     std::string msg;
                     msg.append("Frames written to the file");
                     _messageFld->setStringValue(msg.c_str());
                 }
                    // for (;out_size; i++)
                    // {
                          out_size = avcodec_encode_video(c, outbuf, outbuf_size, NULL); 
                          std::string msg;                       
                          msg.append("Writing Frames");
                          _messageFld->setStringValue(msg.c_str());// encode the delayed frames
                          _numFramesFld->setIntValue(_numFramesFld->getIntValue()+1);
                          fwrite(outbuf, 1, out_size, f);
                    // }
                     outbuf[0] = 0x00;
                     outbuf[1] = 0x00;                                                                                               // add sequence end code to have a real mpeg file
                     outbuf[2] = 0x01;
                     outbuf[3] = 0xb7;
                     fwrite(outbuf, 1, 4, f);
}

Then close and clean the Image Buffer and file

  ML_TRACE_IN("MovieCreator::_endRecording()")
if (_numFramesFld->getIntValue() == 0)
{
    _messageFld->setStringValue("Empty movie, nothing saved.");
} 
else 
{
    _messageFld->setStringValue("Movie written to disk.");
    _numFramesFld->setIntValue(0);
if (_hbitmap)
{ 
    DeleteObject(_hbitmap); 
}
if (c != NULL)
{
       av_free(outbuffer);     
       av_free(inpic);
       av_free(outpic);
       fclose(f);
       avcodec_close(c);                                                                                               // freeing memory
       free(outbuf);
       av_free(c);
}
}

}

I think the Main Problem is over here !!

                     //out_size = avcodec_encode_video(c, outbuf,outbuf_size, outpic);

Answer 1

H264 can't read in RGB frames. Change this line from this:

    c->pix_fmt = AV_PIX_FMT_RGB32

To this:

    c->pix_fmt = AV_PIX_FMT_YUV420P

Also specify less inputs into the coder, let FFMPEG decide the best settings. Remove these lines and see if it then works:

    c->i_quant_factor = (float)0.71; 
    c->max_b_frames = 1;
    c->keyint_min = 1;

Alternatively, you could also try starting from this working code example here:

http://www.imc-store.com.au/Articles.asp?ID=276

The example is in VS2010 and is using FFMPEG to encode frames into a H264 encoded AVI file. It has plenty of comments and I've found it really helpful.

You can pass the BGR char array that you read from your Bitmap into the FFMPEG class (just swap the colours to RGB).