iPhone 中 H 264 stream 的 SPS 值

Question

Can someone point me to documentation that will help me get correct SPS and PPS values for iPhone.

Answer 1

Question is a bit unclear...

Picture Parameter Set is described in the latest ITU-T release of the standard in chapter 7.3.2.2

Sequence Parameter Set is described in chapter 7.3.2.1.

Answer 2

I am sure you know, but you can only save H264 encoded video into a file(.mp4, .mov) on iOS. There is no access to encoded video frames from code yet. So if you want to create an mp4 file that has encoded video, you need to use AVAssetWriter. Apple has a good example code on how to do this.

I don't know of any place where different SPS/PPS are published; since they vary based on your compression settings, image size, and whether you are encoding video in portrait or landscape mode. You can use above example code(RosyWriter) to generate some small .mp4 files with your encoding presets; and then I would use a hex editor to find the SPS/PPS manually. Note that SPS/PPS will towards on the end of the file after your H264 stream as part of a larger mp4 info structure. You can find more info on its structure online.

Here is some SPS/PPS that I found useful for my project. Some of them might work for you, but if not you can always generate an mp4 with your H264 encoding presets and find necessary SPS/PPS. My video was encoded using AVVideoProfileLevelH264Baseline30, and here are the SPS/PPS for different video sizes I needed:

SPS:

// For AVCaptureSessionPresetLow(144x192) AVCaptureSessionLandscape on Iphone4S, Iphone5
char iphone_sps[] = {0x67, 0x4D, 0x00, 0x0C, 0xAB, 0x41, 0x82, 0x74, 0xD4, 0x04, 0x04, 0x18, 0x08};

// For AVCaptureSessionPresetLow(144x192), AVCaptureVideoOrientationPortrait on all Ipads
char ipad_sps[] = {0x67, 0x4D, 0x00, 0x0C, 0xAB, 0x41, 0x23, 0x34, 0xD4, 0x04, 0x04, 0x18, 0x08};

// Iphone 4G AVCaptureSessionPresetLow (144x192), AVCaptureVideoOrientationPortrait
char iphone4g_sps[] = {0x67, 0x42, 0x00, 0x1E, 0x8D, 0x68, 0x24, 0x66, 0x9A, 0x83, 0x00, 0x83, 0x01};

// For AVCaptureSessionPreset352x288 (352x288), AVCaptureVideoOrientationLandscape 
char iphone_sps[] = {0x67, 0x42, 0x00, 0x1E, 0xAB, 0x40, 0xB0, 0x4B, 0x4D, 0x40, 0x40, 0x41, 0x80, 0x80};

// For AVCaptureSessionPreset352x288 (352x288), AVCaptureVideoOrientationPortrait
char ipad_sps[] = {0x67, 0x42, 0x00, 0x1E, 0xAB, 0x40, 0xB0, 0x4B, 0x4D, 0x40, 0x40, 0x41, 0x80, 0x80};

PPS:

char pps[] =  {0x28, 0xCE, 0x3C, 0x80};
char iphone4g_pps[] = {0x68, 0xCE, 0x09, 0xC8};

Answer 3

You can encode a single frame to a file and then extract the sps and pps from that file. I have an example that shows how to do exactly that at http://www.gdcl.co.uk/2013/02/20/iOS-Video-Encoding.html

Answer 4

Use VideoToolbox API. refer: https://developer.apple.com/videos/play/wwdc2014/513/

Search keyword print("SPS is and print("PPS is below.

//
//  LiveStreamSession.swift
//  LiveStreamKit
//
//  Created by Ya Wang on 6/10/21.
//

import Foundation
import AVFoundation
import VideoToolbox

public class LiveStreamSession {
    
    let compressionSession: VTCompressionSession
        
    var index = -1
    
    var lastInputPTS = CMTime.zero
    
    public init?(width: Int32, height: Int32){
        var compressionSessionOrNil: VTCompressionSession? = nil
        let status = VTCompressionSessionCreate(allocator: kCFAllocatorDefault,
                                                width: width,
                                                height: height,
                                                codecType: kCMVideoCodecType_H264,
                                                encoderSpecification: nil, // let the video toolbox choose a encoder
                                                imageBufferAttributes: nil,
                                                compressedDataAllocator: kCFAllocatorDefault,
                                                outputCallback: nil,
                                                refcon: nil,
                                                compressionSessionOut: &compressionSessionOrNil)
        guard status == noErr,
            let compressionSession = compressionSessionOrNil else {
            return nil
        }
        VTSessionSetProperty(compressionSession, key: kVTCompressionPropertyKey_RealTime, value: kCFBooleanTrue);
        VTCompressionSessionPrepareToEncodeFrames(compressionSession)
        
        self.compressionSession = compressionSession
        
    }
    
    public func pushVideoBuffer(buffer: CMSampleBuffer) {
        // image buffer
        guard let imageBuffer = CMSampleBufferGetImageBuffer(buffer) else {
            assertionFailure()
            return
        }
        
        // pts
        let pts = CMSampleBufferGetPresentationTimeStamp(buffer)
        guard CMTIME_IS_VALID(pts) else {
            assertionFailure()
            return
        }
        
        // duration
        var duration = CMSampleBufferGetDuration(buffer);
        if CMTIME_IS_INVALID(duration) && CMTIME_IS_VALID(self.lastInputPTS) {
            duration = CMTimeSubtract(pts, self.lastInputPTS)
        }
                
        index += 1
        self.lastInputPTS = pts
        print("[\(Date())]: pushVideoBuffer \(index)")
        
        let currentIndex = index
        VTCompressionSessionEncodeFrame(compressionSession, imageBuffer: imageBuffer, presentationTimeStamp: pts, duration: duration, frameProperties: nil, infoFlagsOut: nil) {[weak self] status, encodeInfoFlags, sampleBuffer in
            print("[\(Date())]: compressed \(currentIndex)")
            if let sampleBuffer = sampleBuffer {
                self?.didEncodeFrameBuffer(buffer: sampleBuffer, id: currentIndex)
            }
        }
    }
    
    deinit {
        VTCompressionSessionInvalidate(compressionSession)
    }
    
    private func didEncodeFrameBuffer(buffer: CMSampleBuffer, id: Int) {
        guard let attachments = CMSampleBufferGetSampleAttachmentsArray(buffer, createIfNecessary: true)
               else {
            return
        }
        let dic = Unmanaged<CFDictionary>.fromOpaque(CFArrayGetValueAtIndex(attachments, 0)).takeUnretainedValue()
        let keyframe = !CFDictionaryContainsKey(dic, Unmanaged.passRetained(kCMSampleAttachmentKey_NotSync).toOpaque())
//        print("[\(Date())]: didEncodeFrameBuffer \(id) is I frame: \(keyframe)")
        if keyframe,
           let formatDescription = CMSampleBufferGetFormatDescription(buffer) {
            // https://www.slideshare.net/instinctools_EE_Labs/videostream-compression-in-ios
            var number = 0
            CMVideoFormatDescriptionGetH264ParameterSetAtIndex(formatDescription, parameterSetIndex: 0, parameterSetPointerOut: nil, parameterSetSizeOut: nil, parameterSetCountOut: &number, nalUnitHeaderLengthOut: nil)
            // SPS and PPS and so on...
            let parameterSets = NSMutableData()
            for index in 0 ... number - 1 {
                var parameterSetPointer: UnsafePointer<UInt8>?
                var parameterSetLength = 0
                CMVideoFormatDescriptionGetH264ParameterSetAtIndex(formatDescription, parameterSetIndex: index, parameterSetPointerOut: &parameterSetPointer, parameterSetSizeOut: &parameterSetLength, parameterSetCountOut: nil, nalUnitHeaderLengthOut: nil)
//                parameterSets.append(startCode, length: startCodeLength)
                if let parameterSetPointer = parameterSetPointer {
                    parameterSets.append(parameterSetPointer, length: parameterSetLength)
                }
                
                //
                if index == 0 {
                    print("SPS is \(parameterSetPointer) with length \(parameterSetLength)")
                } else if index == 1 {
                    print("PPS is \(parameterSetPointer) with length \(parameterSetLength)")
                }
            }
            print("[\(Date())]: parameterSets \(parameterSets.length)")
        }
    }
}