Read Second audio track stream from mp4 file using SharpDx or IMSourceReader

Question

I have a requirement in my application where I have to read all the available track stream from mp4 file.

Mp4 file is encoded with number of tracks in AAC format. I have to decode to get all available tracks from the file. Currently I am using SharpDX and IMSourceReader (Media Foundation dlls) to read the Streams. But by default SourceReader returns only the first audio stream from the file. Is it I am doing correct ? Or I have to use any other third party libraries to achieve this ?

Answer 1

When configuring the reader , you can select which streams will be delivered when reading samples. Often times you do not wish to select the stream. An example would be a movie which has additional audio streams (spanish, french, or perhaps director commentary). As a result, most of the time stream selection is as simple as the following:

// error checking omitted for brevity
hr = reader->SetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, nullptr, audioMediaType);
hr = reader->SetStreamSelection((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, true);

However if you look at SetStreamSelection , the first parameter takes either the enumeration used above, or a specific stream index.

// 0–0xFFFFFFFB  <-- The zero-based index of a stream.
//   0xFFFFFFFC  <-- MF_SOURCE_READER_FIRST_VIDEO_STREAM
//   0xFFFFFFFD  <-- MF_SOURCE_READER_FIRST_AUDIO_STREAM
//   0xFFFFFFFE  <-- MF_SOURCE_READER_ALL_STREAMS
//   0xFFFFFFFE  <-- MF_SOURCE_READER_ANY_STREAM
//   0xFFFFFFFF  <-- MF_SOURCE_READER_INVALID_STREAM_INDEX

I have never used SharpDX, but this enumeration is documented here . Pertaining to video, sometimes additional video streams are available (usually closed captioning).

When reading the samples, using a callback or synchronously , pay close attention to the stream index, and process the sample accordingly.

You may also find these answers valuable or interesting:
Aggregate Media Source
MP4 IMFSinkWriter
Adding Audio Sample to Video
Creating NV12 Encoded Video
IMFSinkWriter Configuration
IMFSinkWriter CPU Utilization

I hope this helps.