[英]Play audio from CMSampleBuffer
我为 iOS 中的群组创建了一个视频聊天应用程序。 我一直在寻找一些方法来分别控制不同参与者的音量。 我找到了在RemoteAudioTrack
使用isPlaybackEnabled
静音和取消静音的方法,但不能控制音量。
我还想我们是否可以在AVAudioPlayer
使用它。 我找到了addSink
。 这是我从这里尝试的:
class Audio: NSObject, AudioSink {
var a = 1
func renderSample(_ audioSample: CMSampleBuffer!) {
print("audio found", a)
a += 1
var audioBufferList = AudioBufferList()
var data = Data()
var blockBuffer : CMBlockBuffer?
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(audioSample, bufferListSizeNeededOut: nil, bufferListOut: &audioBufferList, bufferListSize: MemoryLayout<AudioBufferList>.size, blockBufferAllocator: nil, blockBufferMemoryAllocator: nil, flags: 0, blockBufferOut: &blockBuffer)
let buffers = UnsafeBufferPointer<AudioBuffer>(start: &audioBufferList.mBuffers, count: Int(audioBufferList.mNumberBuffers))
for audioBuffer in buffers {
let frame = audioBuffer.mData?.assumingMemoryBound(to: UInt8.self)
data.append(frame!, count: Int(audioBuffer.mDataByteSize))
}
let player = try! AVAudioPlayer(data: data) //crash here
player.play()
}
}
但是它在let player = try! AVAudioPlayer(data: data)
上崩溃了let player = try! AVAudioPlayer(data: data)
let player = try! AVAudioPlayer(data: data)
。
编辑:
这是错误: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=NSOSStatusErrorDomain Code=-39 "(null)": file
Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=NSOSStatusErrorDomain Code=-39 "(null)": file
。
这是data
所以我想它没有转换:
▿ 0 bytes
- count : 0
▿ pointer : 0x000000016d7ae160
- pointerValue : 6131736928
- bytes : 0 elements
这是audioSample
:
<CMAudioFormatDescription 0x2815a3de0 [0x1bb2ef830]> {
mediaType:'soun'
mediaSubType:'lpcm'
mediaSpecific: {
ASBD: {
mSampleRate: 16000.000000
mFormatID: 'lpcm'
mFormatFlags: 0xc
mBytesPerPacket: 2
mFramesPerPacket: 1
mBytesPerFrame: 2
mChannelsPerFrame: 1
mBitsPerChannel: 16 }
cookie: {(null)}
ACL: {(null)}
FormatList Array: {(null)}
}
extensions: {(null)}
}
您可以从CMSampleBuffer获取完整的数据缓冲区,并将其转换为Data :
let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer)
let blockBufferDataLength = CMBlockBufferGetDataLength(blockBuffer!)
var blockBufferData = [UInt8](repeating: 0, count: blockBufferDataLength)
let status = CMBlockBufferCopyDataBytes(blockBuffer!, atOffset: 0, dataLength: blockBufferDataLength, destination: &blockBufferData)
guard status == noErr else { return }
let data = Data(bytes: blockBufferData, count: blockBufferDataLength)
还可以参考AVAudioPlayer概述:
除非要播放从网络流捕获的音频或需要非常低的I / O延迟,否则请使用此类进行音频播放。
所以我认为这不会为您服务。 您最好使用AVAudioEngine或音频队列服务 。
虽然我在为什么你的方法都不奏效不确定,我已经编辑等CMSampleBuffer
S使用Data
。 尝试这个:
class Audio: NSObject {
func renderSample(_ sampleBuffer: CMSampleBuffer!) {
let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
CVPixelBufferLockBaseAddress(imageBuffer!, CVPixelBufferLockFlags(rawValue: 0))
let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!)
let height = CVPixelBufferGetHeight(imageBuffer!)
let src_buff = CVPixelBufferGetBaseAddress(imageBuffer!)
let data = Data(bytes: src_buff!, count: bytesPerRow * height)
CVPixelBufferUnlockBaseAddress(imageBuffer!, CVPixelBufferLockFlags(rawValue: 0))
doSomething(data: Data)
}
func doSomething(data:Data)
{
print(data.count) //Make sure isn't 0
}
}
这只是从缓冲区创建Data
,并确保锁定和解锁,以使其他人不锁定或不锁定。 由于我没有要测试的音频缓冲区,所以这只是黑暗中的一枪。 让我知道!
尝试将音频文件保存到文档目录,然后播放声音。 这对我有用。
func playMusic() {
let url = NSBundle.mainBundle().URLForResource("Audio", withExtension: "mp3")!
let data = NSData(contentsOfURL: url)!
AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, error: nil)
AVAudioSession.sharedInstance().setActive(true, error: nil)
audioPlayer = AVAudioPlayer(data: data, fileTypeHint: AVFileTypeMPEGLayer3, error: nil)
audioPlayer.prepareToPlay()
audioPlayer.play()
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.