Read raw Genicam H.264 data to avlib

Question

I try to get familiar with libav in order to process a raw H.264 stream from a GenICam supporting camera. I'd like to receive the raw data via the GenICam provided interfaces (API), and then forward that data into libav in order to produce a container file that then is streamed to a playing device like VLC or (later) to an own implemented display.

So far, I played around with the GenICam sample code, which transferres the raw H.264 data into a "sample.h264" file. This file, I have put through the command line tool ffmpeg, in order to produce an mp4 container file that I can open and watch in VLC

command: ffmpeg -i "sample.h264" -c:v copy -f mp4 "out.mp4"

Currently, I dig through examples and documentations for each H.264, ffmpeg, libav and video processing in general. I have to admit, as total beginner, it confuses me a lot. I'm at the point where I think I have found the according libav functions that would help my undertaking:

I think, basically, I need the functions avcodec_send_packet() and avcodec_receive_packet() (since avcodec_decode_video2() is deprecated). Before that, I set up an avCodedContext structure and open (or combine?!?) it with the H.264 codec (AV_CODEC_ID_H264).

So far, my code looks like this (omitting error checking and other stuff):

...
AVCodecContext* avCodecContext = nullptr;
AVCodec *avCodec = nullptr;
AVPacket *avPacket = av_packet_alloc();
AVFrame *avFrame = nullptr;
...
avCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
avCodecContext = avcodec_alloc_context3(avCodec);
avcodec_open2 ( avCodecContext, avCodec, NULL );
av_init_packet(avPacket);
...

while(receivingRawDataFromCamera)
{
  ...
  // receive raw data via GenICam
  DSGetBufferInfo<void*>(hDS, sBuffer.BufferHandle, BUFFER_INFO_BASE, NULL, pPtr)

  // libav action
  avPacket->data =static_cast<uint8_t*>(pPtr);  
  avErr = avcodec_send_packet(avCodecContext, avPacket);
  avFrame = av_frame_alloc();
  avErr = avcodec_receive_frame( avCodecContext, avFrame);

  // pack frame in container? (not implemented yet)
  ..
}

The result of the code above is, that both calls to send_packet() and receive_frame() return error codes (-22 and -11), which I'm not able to decrypt via av_strerror() (it only says, these are error codes 22 and 11).

Edit: Maybe as an additional information for those who wonder if

avPacket->data = static_cast<uint8_t*>(pPtr);

is a valid operation... After the very first call to this operation, the content of avPacket->data is

{0x0, 0x0, 0x0, 0x1, 0x67, 0x64, 0x0, 0x28, 0xad, 0x84, 0x5,
  0x45, 0x62, 0xb8, 0xac, 0x54, 0x74, 0x20, 0x2a, 0x2b, 0x15, 0xc5,
  0x62}

which somehow looks as something to be expected becaus of the NAL marker and number in the beginning? I don't know, since I'm really a total beginner....

The question now is, am I on the right path? What is missing and what do the codes 22 and 11 mean?

The next question would be, what to do afterwards, in order to get a container that I can stream (realtime) to a player?

Thanks in advance, Maik

Answer 1

At least for the initally asked question I found the solution for myself:

In order to get rid of the errors on calling the functions

avcodec_send_packet(avCodecContext, avPacket);
...
avcodec_receive_frame( avCodecContext, avFrame);

I had to manually fill some parameters of 'avCodecContext' and 'avPacket':

avCodecContext->bit_rate = 8000000;
avCodecContext->width = 1920;
avCodecContext->height = 1080;
avCodecContext->time_base.num = 1;
avCodecContext->time_base.den = 25;
...
avPacket->data = static_cast<uint8_t*>(pPtr);
avPacket->size = datasize;
avPacket->pts = frameid;

whereas 'datasize' and 'frameid' are received via GenICam, and may not be the appropriate parameters for the fields, but at least I do not get any errors anymore.

Since this answers my initial question on how I get the raw data into the structures of libav, I think, the question is answered.

The discussion and suggestions with/from Vencat in the commenst section lead to additional questions I have, but which should be discussed in a new question, I guess.