How to interpret the pixel array derived from CMSampleBuffer in Swift

Question

Maybe this is a very stupid question. I am using AVFoundation in my app and I am able to get the frames(32BGRA Format). The width of the frame is 1504, Height is 1128 and the bytes-Per-Row value is 6016. When I create a UInt8 pixel array from this samplebuffer the length (array.count) of this array is 1696512 which happens to be equal to width * height.

What I am not getting is why the array length is width * height. Should it not be width * height * 4.

What am I missing here?

Edit - 1: Code

func BufferToArray(sampleBuffer: CMSampleBuffer) -> ([UInt8], Int, Int, Int) {

    var rgbBufferArray = [UInt8]()

    //Get pixel Buffer from CMSSampleBUffer
    let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!

    //Lock the base Address
    CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags.readOnly)

    let width = CVPixelBufferGetWidth(pixelBuffer)
    let height = CVPixelBufferGetHeight(pixelBuffer)
    //get pixel count
    let pixelCount = CVPixelBufferGetWidth(pixelBuffer) * CVPixelBufferGetHeight(pixelBuffer)

    //Get base address
    let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)

    //Get bytes per row of the image
    let bytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)

    //Cast the base address to UInt8. This is like an array now
    let frameBuffer = baseAddress?.assumingMemoryBound(to: UInt8.self)


    rgbBufferArray = Array(UnsafeMutableBufferPointer(start: frameBuffer, count: pixelCount))


    //Unlock and release memory
    CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))

return (rgbBufferArray, bytesPerRow, width, height)

}

Answer 1

The culprit is the data type ( UInt8 ) in combination with the count :

You are assuming the memory contains UInt8 values ( assumingMemoryBound(to: UInt8.self) ) of pixelCount count. But as you concluded correctly it should be four times that number.

I'd recommend you import simd and use simd_uchar4 as data type. That's a struct type containing 4 UInt8 . Then your array will contain pixelCount values of 4-tuple pixel values. You can access the channels with array[index].x , .y , .z , and .w respectively.