通過 tcp 套接字流式傳輸 PCM 音頻

Question

我有來自 TCP 套接字的連續原始 PCM 音頻數據流，我想播放它們。 我做了很多研究，看到了很多樣本​​，但沒有結果。 這個要點是最接近的解決方案，但問題是，它是流式 mp3 文件。 所以我有一個套接字，它接收線性 PCM 音頻數據並將它們提供給播放器，如下所示：

func play(_ data: Data) {
    // this function is called for every 320 bytes of linear PCM data.
    // play the 320 bytes of PCM data here!
}

那么是否有任何“簡單”的方式來播放原始 PCM 音頻數據？

Answer 1

對於 iOS，您可以使用 RemoteIO 音頻單元或帶有循環緩沖區的 AVAudioEngine 來實現實時音頻流。

您不能將網絡數據直接提供給音頻輸出，而應將其放入循環緩沖區中，音頻子系統播放回調可以從中以固定速率使用它。 您將需要預先緩沖一定數量的音頻樣本以覆蓋網絡抖動。

這樣做的簡單“方法”可能無法優雅地處理網絡抖動。

Answer 2

回答晚了，但如果您仍然堅持播放 TCP 字節，請嘗試按照我的回答將 tcp 音頻字節放入循環緩沖區並通過 AudioUnit 播放。 下面的代碼從 TCP 接收字節並將它們放入 TPCircularBuffer

func tcpReceive() {
        receivingQueue.async {
            repeat {
                do {
                    let datagram = try self.tcpClient?.receive()
                    var byteData = datagram?["data"] as? Data
                    let dataLength = datagram?["length"] as? Int


                    let _ = TPCircularBufferProduceBytes(&self.circularBuffer, byteData.bytes, UInt32(decodedLength * 2))

                } catch {
                    fatalError(error.localizedDescription)
                }
            } while true
        }
    }

創建音頻單元...

var desc = AudioComponentDescription(
            componentType: OSType(kAudioUnitType_Output),
            componentSubType: OSType(kAudioUnitSubType_VoiceProcessingIO),
            componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
            componentFlags: 0,
            componentFlagsMask: 0
        )

        let inputComponent = AudioComponentFindNext(nil, &desc)

        status = AudioComponentInstanceNew(inputComponent!, &audioUnit)
        if status != noErr {
            print("Audio component instance new error \(status!)")
        }

 // Enable IO for playback
        status = AudioUnitSetProperty(
            audioUnit!,
            kAudioOutputUnitProperty_EnableIO,
            kAudioUnitScope_Output,
            kOutputBus,
            &flag,
            SizeOf32(flag)
        )
        if status != noErr {
            print("Enable IO for playback error \(status!)")
        }

//Use your own format, I have sample rate of 16000 and pcm 16 Bit
        var ioFormat = CAStreamBasicDescription(
            sampleRate: 16000.0,
            numChannels: 1,
            pcmf: .int16,
            isInterleaved: false
        )
    

//This is playbackCallback 
    var playbackCallback = AURenderCallbackStruct(
            inputProc: AudioController_PlaybackCallback, //This is a delegate where audioUnit puts the bytes
            inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
        )

        status = AudioUnitSetProperty(
            audioUnit!,
            AudioUnitPropertyID(kAudioUnitProperty_SetRenderCallback),
            AudioUnitScope(kAudioUnitScope_Input),
            kOutputBus,
            &playbackCallback,
            MemoryLayout<AURenderCallbackStruct>.size.ui
        )
        if status != noErr {
            print("Failed to set recording render callback \(status!)")
        }
//Init Audio Unit
  status = AudioUnitInitialize(audioUnit!)
        if status != noErr {
            print("Failed to initialize audio unit \(status!)")
        }

//Start AudioUnit
     status = AudioOutputUnitStart(audioUnit!)
        if status != noErr {
            print("Failed to initialize output unit \(status!)")
        }
    
    

這是我的 playbackCallback 函數，我在其中播放循環緩沖區中的音頻

func performPlayback(
        _ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>,
        inTimeStamp: UnsafePointer<AudioTimeStamp>,
        inBufNumber: UInt32,
        inNumberFrames: UInt32,
        ioData: UnsafeMutablePointer<AudioBufferList>
    ) -> OSStatus {
        let buffer = ioData[0].mBuffers

        let bytesToCopy = ioData[0].mBuffers.mDataByteSize
        var bufferTail: UnsafeMutableRawPointer?

        var availableBytes: UInt32 = 0
        bufferTail = TPCircularBufferTail(&self.circularBuffer, &availableBytes)
        let bytesToWrite = min(bytesToCopy, availableBytes)


        var bufferList = AudioBufferList(
            mNumberBuffers: 1,
            mBuffers: ioData[0].mBuffers)

        var monoSamples = [Int16]()
        let ptr = bufferList.mBuffers.mData?.assumingMemoryBound(to: Int16.self)
        monoSamples.append(contentsOf: UnsafeBufferPointer(start: ptr, count: Int(inNumberFrames)))
        print(monoSamples)

        memcpy(buffer.mData, bufferTail, Int(bytesToWrite))
        TPCircularBufferConsume(&self.circularBuffer, bytesToWrite)

        return noErr
    }

對於 TPCircularBuffer 我使用了這個 pod

'TPCircularBuffer', '~> 1.6'

Answer 3

所有詳細說明和示例代碼均可用於

音頻工具箱/音頻單元

您可以注冊回調以從 AUGraph 獲取 PCM 數據並將 pcm 緩沖區發送到套接字。

更多用法示例：

https://github.com/rweichler/coreaudio-examples/blob/master/CH08_AUGraphInput/main.cpp