How is frame data stored in libav?

Question

I am trying to learn to use libav. I have followed the very first tutorial on dranger.com, but I got a little confused at one point.

// Write pixel data
for(y=0; y<height; y++)
fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile);

This code clearly works, but I don't quite understand why, particulalry I don't understand how the frame data in pFrame->data stored, whether or not it depends on the format/codec in use, why pFrame->data and pFrame->linesize is always referenced at index 0, and why we are adding y to pFrame->data[0] .

In the tutorial it says

We're going to be kind of sketchy on the PPM format itself; trust us, it works.

I am not sure if writing it to the ppm format is what is causing this process to seem so strange to me. Any clarification on why this code is the way it is and how libav stores frame data would be very helpful. I am not very familiar with media encoding/decoding in general, thus why I am trying to learn.

Answer 1

particularly I don't understand how the frame data in pFrame->data stored, whether or not it depends on the format/codec in use

Yes, It depends on the pix_fmt value. Some formats are planar and others are not.

why pFrame->data and pFrame->linesize is always referenced at index 0,

If you look at the struct, you will see that data is an array of pointers/a pointer to a pointer. So pFrame->data[0] is a pointer to the data in the first "plane". Some formats, like RGB have a singe plane, where all data is stored in one buffer. Other formats like YUV, use a separate buffer for each plane. eg Y = pFrame->data[0], U = pFrame->data[1], pFrame->data[3] Audio may use one plane per channel, etc.

and why we are adding y to pFrame->data[0].

Because the example is looping over an image line by line, top to bottom. To get the pointer to the fist pixel of any line, you multiply the linesize by the line number then add it to the pointer.