Distorted audio when converting PCM to AAC using AudioConverter and writing to .mp4 file using AVAssetWriter

Question

I'm trying to compress input from AVCaptureSession and save it to an .mp4 file. I've managed to convert and save video but am having trouble with audio.

I've reached a point where I'm using https://github.com/nevyn/LBMediaToolkit to convert audio. I can save the converted CMSampleBuffer to .mp4 but when I play it back on the iPhone, the audio sounds distorted.

The weird thing is that the audio also sounds distorted when played in QuickTime, but it sounds OK when played through a web browser. Click here to download a sample file.

I think I'm passing the correct format to the converter:

let asbd = AudioStreamBasicDescription(
    mSampleRate: 44_100,
    mFormatID: kAudioFormatMPEG4AAC,
    mFormatFlags: UInt32(MPEG4ObjectID.AAC_LC.rawValue),
    mBytesPerPacket: 0,
    mFramesPerPacket: 1024,
    mBytesPerFrame: 0,
    mChannelsPerFrame: 2,
    mBitsPerChannel: 0,
    mReserved: 0)

return LBAudioConverter(convertingTo: asbd)

I'm not sure if it's an issue with audio conversion or writing the file. Here's a link to a sample project with just the audio: https://github.com/gaperlinski/AudioConversion (follow the instructions in README.MD to reproduce the issue)

Is there anything obvious I'm doing wrong that causes the audio to sound distorted?

Answer 1

It turns out that the culprit was the following code:

UInt32 cookieSize = 0;
char cookie[cookieSize];

AudioConverterGetPropertyInfo(_converter, kAudioConverterCompressionMagicCookie, &cookieSize, NULL);
AudioConverterGetProperty(_converter, kAudioConverterCompressionMagicCookie, &cookieSize, cookie);

CMAudioFormatDescriptionRef audioFormat;
const OSStatus formatCreationError = CMAudioFormatDescriptionCreate(
    kCFAllocatorDefault,
    &_toFormat,
    0, NULL, // layout
    cookieSize, cookie, // cookie
    NULL, // extensions
    &audioFormat);
if(formatCreationError != noErr) {
    NSLog(@"Failed to convert a buffer because format creation failed: %d", (int)formatCreationError);
    continue;
}

I've noticed that when I create the format description for the CMSampleBuffer, I should add the magicCookie only to the first sample buffer , or perhaps it should be sent even in a sample buffer with no data at all.

If I do it that way, there's no distortion in the audio and there's only one esds atom in the mp4 file container.