using metadata or side_data in AVFrame / AVPacket to store a number

Question

I'm struggling with adding a number to a AVFame/AVPacket before I encode it and retrieve the number back when I decode it. My original question was here but I wasn't able to get it work with the AVFrame metadata or side_data. I have looked at various posts in stackflow like this or on the inte.net ( using AVDictionary, AVDictionaryEntry... ) but nothing so far. I have managed to store a number in the metadata object of the frame but it is not there when I decode the package. Anyone have an idea what I need to do? Are my coding settings not correct and is therefor my custom data not available when I decode the packet/frame?

Answer 1

If you're able to use MKV as the container format, you can attach the number as BlockAdditional side-data to the AVPacket , and retrieve it from the AVPacket that is read from the container during playback. Here is a crude example, using strings since they are easy to read.

Note: if you want to correlate the side-data to AVFrame s, you will probably need to do some additional tracking to match each AVPacket to the corresponding AVFrame . In FFmpeg, the codec will not preserve the side-data through decoding.

AVPacket* packet;     // from avcodec_receive_packet
int64_t frame_number; // number you want to attach

// Allocate the side data on the packet.
size_t side_data_size = 256;
uint8_t* side_data = av_packet_new_side_data(
    packet, AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL, side_data_size);

// Set the BlockAddID. Currently only 1 is supported.
// BlockAdditional data with BlockAddID==1 is intended to store
// data for the codec to use.
uint64_t additional_id = 1;
AV_WB64(side_data, additional_id);

// Put your data in the side_data after the ID.
std::string msg = "frame_number=" + std::to_string(frame_number);
std::strcpy(reinterpret_cast<char*>(side_data + 8), msg.c_str());

Then during playback, you can retrieve the side-data like this:

AVPacket* packet;     // from av_read_frame

// Get side-data from packet.
int side_data_size;
uint8_t* side_data = av_packet_get_side_data(
    &packet, AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL, &side_data_size);

// Print it, or parse it how you want.
std::string msg = (char*)(side_data + 8);
std::cout << msg << std::endl;