FFmpeg av_read_frame returns packets from audio stream

Question

I am currently trying to learn the FFmpeg API , following this tutorial . However, I already have issues with the first lesson on video decoding. My code is basically the same as the one from the tutorial except I am using C++. My issue is that the video stream does not match the one from the packet returned by av_read_frame .

The video stream is obtained looping on the available streams until the video stream is found.

for(int i = 0; i < pFormatCtx->nb_streams; i++) { // nb_streams == 2

    if(pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO) {
        videoStream = i; 
        break; // videoStream == 0
    }
}

Then when retrieving the frame data, it seams grabbing the audio channel.

while(av_read_frame(pFormatCtx, &packet) >= 0) { // read returns 0

    // Is this a packet from the video stream?
    if(packet.stream_index == videoStream) { 
        //packet.stream_index == 1, which correspond to the audio stream
    } 
}

I have not found examples online where this test is actually failing. Have I miss some way to specify the stream_index that is not in the tutorial? Maybe the tutorial is not up to date and is doing something wrong? If so, what is the correct way to extract the frame data? In case that matters, I am using the latest FFmpeg 4.0.2 build, on Windows 64-bits , compiling with Visual Studio 2017 .

On videos with no sound, the two streams match and I am able to decode and display the frames correctly.

Answer 1

Try like this:

while(av_read_frame(pFormatCtx, &packet) == 0) {

    AVStream *st = pFormatCtx->streams[packet.stream_index];
    switch (st->codecpar->codec_type)
    {
        case AVMEDIA_TYPE_AUDIO:
            /* handle audio */
            break;

        case AVMEDIA_TYPE_VIDEO:
            /* handle video */
            break;

        ...
    }
}