ffmpeg H264 Encode Frame at a time for network streaming

Question

I'm working on a remote desktop application, I would like to send an encoded H264 packet over TCP by using ffmpeg for the encoding. However I couldn't find useful info for the particular case of encoding just one frame (already on YUV444) and get the packet.

I have several issues, the first was that:

avcodec_encode_video2

Was not blocking, I found that most of the time you get the "delayed" frames at the end, however, since this is a real time streaming the solution was:

av_opt_set(mCodecContext->priv_data, "tune", "zerolatency", 0);

Now I got the frame, but several issues, it takes a while and even worse I got a gray with trash pixels video as result. My configuration for the Codec Context:

 m_pCodecCtx->bit_rate=8000000;
m_pCodecCtx->codec_id=AV_CODEC_ID_H264;
m_pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
m_pCodecCtx->width=1920;
m_pCodecCtx->height=1080;
m_pCodecCtx->pix_fmt=AV_PIX_FMT_YUV444P;
m_pCodecCtx->time_base.num = 1;
m_pCodecCtx->time_base.den = 25;
m_pCodecCtx->gop_size = 1;
m_pCodecCtx->keyint_min = 1;
m_pCodecCtx->i_quant_factor = float(0.71);
m_pCodecCtx->b_frame_strategy = 20;
m_pCodecCtx->qcompress = (float)0.6;
m_pCodecCtx->qmax = 51;
m_pCodecCtx->qmin = 20;
m_pCodecCtx->max_qdiff = 4;
m_pCodecCtx->refs = 4;
m_pCodecCtx->max_b_frames = 1;
m_pCodecCtx->thread_count = 1;

I would like to know how this could be done, how do I set the "I Frames"? and, that would be the optimal for a "one at a time" encoding? Also I'm not concerned right now with the quality, just need to be fast enough (under 16 ms).

For the encoding part:

nres = avcodec_encode_video2(m_pCodecCtx,&packet,m_pFrame,&framefinished);

if(nres<0){
    qDebug() << "error encoding: " << nres << endl;
}

if(framefinished){
    m_pFrame->pts++;
     ofstream vidout("video.h264",ios::app);
     if(vidout.good()){
         vidout.write((const char*)&packet.data[0],packet.size);
     }
     vidout.close();

     av_packet_unref(&packet);

}

I'm not using a container, just a raw file, ffplay reproduce raw files if the packets are right, and that's my principal issue. I'm planning to send the packet over tcp and decode on the client. Any help would be greatly appreciated.

Answer 1

You could take a look at the source code of webrtc . It use openh264 and ffmpeg to accomplish your work.

I was study in it for a while. But I can't the the latest source code currently.

I found this : source code .

Hope it helps.

Answer 2

Turns out I got it working since the beginning, I made very simple but important mistake, I was writing as text a binary file, so...

Thanks for the feedback and your help