Export 16Bit Grayscale Depth Image (1440x1920) - Swift iOS15.3

Question

I'd like to export a 16 bit grayscale PNG of my captured sceneDepth data on iOS (for machine learning purposes). I am using the new iPadPro with lidar sensor to capture the datra in an ARSession. I already get the 192x256 depth map, which i scaled up by 7.5 to match the 1440x1920 resolution of my rgb images. This is the code i have so far:

func convertDepthToImg(frame: ARFrame) {
        let depthBuffer = frame.sceneDepth?.depthMap
        var ciImageDepth:CIImage    = CIImage(cvPixelBuffer: depthBuffer!) 

        // Transform image on pixel level to get the same size as rgb, apply nearest neighbour sampling or linear sampling (depends on performance in network)   
        let transformation          = CGAffineTransform(scaleX: 7.5, y: 7.5) 
            ciImageDepth            = ciImageDepth.samplingLinear()
                                        .transformed(by: combined_transf_matrix)
        let contextDepth:CIContext  = CIContext(options: nil)  
        let cgImageDepth:CGImage    = contextDepth.createCGImage((ciImageDepth), from: ciImageDepth.extent)!

        // convert to required 16 bits gray png img
        convertTo16BitGrayPng(image: cgImageDepth)
    }

    // Function to create vImageBuffer for more functionality on Images
    func createVImg(image: CGImage) -> vImage_Buffer? {
        guard let vImageBuffer = try? vImage_Buffer(cgImage: image)
        else {
            return nil
        }
        return vImageBuffer
    }

    func convertTo16BitGrayPng(image: CGImage){
        let width = 1440
        let height = 1920

        //create vImageBuffer vor UIImage
        var srcBuf = createVImg(image: image)
        print("Height: ", String(srcBuf!.height))
        print("Width: ", String(srcBuf!.width))

        // allocate memory for final size:
        let bv = malloc(width * height * 4)!
        var db = vImage_Buffer(data: bv,
                               height: vImagePixelCount(height),
                               width: vImagePixelCount(width),
                               rowBytes: width*2)

        // create pointer to Buffer that contains the image data
        vImageConvert_PlanarFtoPlanar16F(&(srcBuf)!, &db, vImage_Flags(kvImageNoFlags))
        let bp = bv.assumingMemoryBound(to: UInt16.self)
        let prov = CGDataProvider(data: CFDataCreateWithBytesNoCopy(kCFAllocatorDefault,
                                                                    bp,
                                                                    height * width * 4,
                                                                kCFAllocatorDefault))!
        let cgImage = CGImage(width: width,
                              height: height,
                              bitsPerComponent: 5,  
                              bitsPerPixel: 16,
                              bytesPerRow: 2 * width, 
                              space: CGColorSpace(name: CGColorSpace.linearSRGB)!, 
                              bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipFirst.rawValue), 
                              provider: prov,
                              decode: nil,
                              shouldInterpolate: false,
                              intent: .defaultIntent)

        // save processed image to documents dir
        saveDptToDocs(cgImage: cgImage!, type: "dpt")
    }

... save Image to documentpath (works fine)

I used this questions answer to convert my images to 16 bit and save them to my documents directory, but i only get 24 bit images. I really can't get the 16 bit export to work. I already exported images in 32, 64, 8 and even 24 bit. However, 16 bit is somewhat tricky? Please help.

Answer 1

If your output image is 16-bit grayscale, I think the colour space you initialise the CGImage should be a grayscale color space (eg CGColorSpaceCreateDeviceGray() ) and the bitsPerComponent and bitsPerPixel should both be 16 . Also, the bitmapInfo should be along the lines of:

CGBitmapInfo.byteOrder16Little.rawValue |
CGBitmapInfo.floatComponents.rawValue |
CGImageAlphaInfo.none.rawValue