Use MediaMuxer to encode a MP4 file but it crashes when calling mediaMuxer.stop()

Question

I'm using MediaMuxer and MediaCodec to encode both Video(H264) data and Audio(AAC-LC) data from camera and audio to a MP4 file. But the MediaMuxer object sometimes crashes at mediaMuxer.stop() and the error log is

11-03 16:28:36.956: A/DEBUG(711): Abort message: 'frameworks/av/media/libstagefright/MPEG4Writer.cpp:2983 CHECK_LT( mCodecSpecificDataSize + 23,128) failed: 399 vs. 128'
11-03 16:28:36.957: A/DEBUG(711):     x0   0000000000000000  x1   0000000000001184  x2   0000000000000006  x3   0000000000000000
11-03 16:28:36.957: A/DEBUG(711):     x4   0000000000000000  x5   0000000000000001  x6   0000000000000000  x7   0000000000000000
11-03 16:28:36.957: A/DEBUG(711):     x8   0000000000000083  x9   0000000000000000  x10  0000000000000000  x11  0000007f91bb0df8
11-03 16:28:36.958: A/DEBUG(711):     x12  0000007f91bb0cd0  x13  0000000000000077  x14  0000007f91bb0ea8  x15  0000000000000000
11-03 16:28:36.958: A/DEBUG(711):     x16  0000007faca8d6a8  x17  0000007faca4fb2c  x18  0000007face14418  x19  0000007f91bb3510
11-03 16:28:36.959: A/DEBUG(711):     x20  0000007f91bb3450  x21  000000000000000b  x22  0000000000000006  x23  00000055a17fd260
11-03 16:28:36.959: A/DEBUG(711):     x24  0000007f91bb1c58  x25  0000007f91bb18b4  x26  0000007f91bb1f90  x27  0000007fa9715000
11-03 16:28:36.960: A/DEBUG(711):     x28  0000007f91bb1898  x29  0000007f91bb0d60  x30  0000007faca4d2c8
11-03 16:28:36.960: A/DEBUG(711):     sp   0000007f91bb0d60  pc   0000007faca4fb34  pstate 0000000020000000

I've tried to only encode a single track(video or audio) many times. The execution of mediaMuxer.stop() was totally fine.

Why CHECK_LT failed when I encoded two tracks?

Answer 1

Ok, I'm here to answer myself.

After working around the clock, I found there are many factors which can cause MediaMuxer.stop() to crash when muxing an AAC track.

In order to avoid the crash, you had better play by the following rules for your implementation:

1. Use MediaCodec in sync mode for AAC encode. By using the sync mode, you can input the audio samples(which is a DirectByteBuffer ) from AudioRecorder to MediaCodec 's inputbuffer quickly without doing any extra memory copy. If you drop too many samples, MediaMuxer will crash.

2. Another reason to use sync mode is that you can use MediaMuxer.dequeueInputBuffer(-1) to make sure you can always get an available InputBuffer in your writeAudioSample method which you implement for write samples to MediaCodec .

3. The other reason to use sync mode is that you need to drain the AAC encoded data before MediaMuxer.stop() . (See the rule 9)

4. Set the MediaFormat.KEY_MAX_INPUT_SIZE by yourself, the value should be a multiple of 2048 (1024 16-bit samples, for bytes, the length is 2048). I usually set it to 8192 that is depends on the audio source's sample rates, the number of channels, and the performance of your applications and devices.

5. Guarantee the input samples is full of audio data, and the length should be the value you set to MediaFormat.KEY_MAX_INPUT_SIZE . And compute the presentation time in microseconds. The interval of presentation time between any two adjacent input samples should be the same. Calculate the duration of your samples by this equation 1_000_000L * KEY_MAX_INPUT_SIZE / SAMPLE_RATE / NUMBER_OF_CHANNELS / 2 . You don't need to align or shift the presentation time to 0 . I strongly recommend you just use long initialTime = System.nanoTime() / 1000L to get the initial time and pass it to MediaCodec . Therefore, the next time you writing the samples, the presentation time should be initialTime + 1_000_000L * KEY_MAX_INPUT_SIZE / SAMPLE_RATE / NUMBER_OF_CHANNELS / 2 and so on.

6. This part is the biggest pit. Check the presentation time when writing the AAC encoded data to MediaMuxer , even though you have already set the presentation time when inputing the audio samples to MediaCodec . The presentation time for the AAC encoded data may not be incremental sometimes. You should record the last presentation time you write to MediaMuxer . If you found the misordered AAC data, adjust the presentation time by presentationTime = ++lastPresentationTime . Also, you may get some zero-presentation-time AAC data. Ignore them. Do not write them to MediaMuxer .

7. If the MediaMuxer has other tracks, make sure the presentation time for each track is in the same range(allowing errors for seconds).

8. One AAC MediaCodec for one MediaMuxer . And do not reuse the output MediaFormat object.

9. Before executing MediaMuxer.stop() , stop calling your writeAudioSample method and send a EOS to MediaCodec via queueInputBuffer(index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM) . And drain the remaining AAC data from MediaCodec to write to MediaMuxer . Execute MediaMuxer.stop() and see what is happened.