AVAudioEngine：过滤实时音频

Question

I have gone through the Apple Sample Code on Equalizing Audio with vDSP , where the audio file is filtered in AVAudioSourceNode and reproduced.

My objective is to do exactly the same, but instead of taking the audio from an audio file, take it in real-time from the microphone. Is it possible to do so in AVAudioEngine ? A couple of ways to do so are based on installTap or AVAudioSinkNode , as described in First strategy and Second strategy sections.

So far, I got a bit closer to my objective with the following 2 strategies.

First strategy

// Added new class variables
private lazy var sinkNode = AVAudioSinkNode { (timestep, frames, audioBufferList) -> OSStatus in
    let ptr = audioBufferList.pointee.mBuffers.mData?.assumingMemoryBound(to: Float.self)
    var monoSamples = [Float]()
    monoSamples.append(contentsOf: UnsafeBufferPointer(start: ptr, count: Int(frames)))
    self.page = monoSamples.
    for frame in 0..<frames {
          print("sink: " + String(monoSamples[Int(frame)]))
        }
    return noErr
}
// AVAudioEngine connections

engine.attach(sinkNode)
// Audio input is passed to the AVAudioSinkNode and the [Float] array is pased to the AVAudioSourceNode through the _page_ variable
engine.connect(input, to: sinkNode, format: formatt)

engine.attach(srcNode)

engine.connect(srcNode,
                   to: engine.mainMixerNode,
                   format: format)

engine.connect(engine.mainMixerNode,
                   to: engine.outputNode,
                   format: format)

// The AVAudioSourceNode access the self.page array through the getSinalElement() function.
private func getSignalElement() -> Float {
    return page.isEmpty ? 0 : page.removeFirst()
}

This approach made it possible to play the audio through the AVAudioSourceNode, but, the audio stops playing after a few seconds (even though, I still successfully get the self.page array in AVAudioSourceNode) and the app finally crashes.

2 strategy

In a similar approach, I used installtap engine.attach(srcNode)

    engine.connect(srcNode,
                   to: engine.mainMixerNode,
                   format: format)

    engine.connect(engine.mainMixerNode,
                   to: engine.outputNode,
                   format: format)
    input.installTap(onBus: 0, bufferSize:1024, format:formatt, block: { [weak self] buffer, when in
        let arraySize = Int(buffer.frameLength)
        let samples = Array(UnsafeBufferPointer(start: buffer.floatChannelData![0], count:arraySize))
        self!.page = samples
    })

// The AVAudioSourceNode access the self.page array through the getSinalElement() function.
private func getSignalElement() -> Float {
    return page.isEmpty ? 0 : page.removeFirst()
}

The outcome after implementing Second strategy is the same as in First strategy . Which can be the issues making these approaches fail?

Answer 1

You can use AvAudioEngine().inputNode like following:

    let engine = AVAudioEngine()
    private lazy var srcNode = AVAudioSourceNode { _, _, frameCount, audioBufferList -> OSStatus in
    
    return noErr
    }
    // Attach First
    engine.attach(srcNode)
    // Then connect nodes

    let input = engine.inputNode

    engine.connect(input, to: srcNode, format: input.inputFormat(forBus: 0))

It is important to use input.inputFormat(...) as format type.

Answer 2

   do{
        try audioSession.setCategory(.playAndRecord, mode: .default, options: [.mixWithOthers, .defaultToSpeaker,.allowBluetoothA2DP,.allowAirPlay,.allowBluetooth])
        try audioSession.setActive(true)
    } catch{
        print(error.localizedDescription)
    }
    
    engine.attach(player)
    //Add this only you want putch
    let pitch = AVAudioUnitTimePitch()
    
   // pitch.pitch = 1000 //Filtered Voice
    //pitch.rate = 1 //Normal rate
    
    // engine.attach(pitch)
    engine.attach(srcNode)
    
    engine.connect(srcNode,
                   to: engine.mainMixerNode,
                   format: engine.inputNode.inputFormat(forBus: 0))
    
    engine.connect(engine.mainMixerNode,
                   to: engine.outputNode,
                   format: engine.inputNode.inputFormat(forBus: 0))
    engine.prepare()
    
    
    engine.inputNode.installTap(onBus: 0, bufferSize: 512, format: engine.inputNode.inputFormat(forBus: 0)) { (buffer, time) -> Void in
        //   self.player.scheduleBuffer(buffer)
        let arraySize = Int(buffer.frameLength)
        let samples = Array(UnsafeBufferPointer(start: buffer.floatChannelData![0], count:arraySize))
        self.page = samples
        print("samples",samples)
    }
    
    
    engine.mainMixerNode.outputVolume = 0.5