Swift 3 CGImage Memory issue

Question

I have this function in my code in Swift3, that I actually almost literally translated from Apple sample code.

My app processes sampleBuffers that comes from capturing live feed from the camera.

This function correctly create and returns an image from a CMSampleBuffer. It works fine, but the memory keeps growing until the apps crash.

With Instruments I saw that there is some image data that doesn't get released. If I comment the line where I do "context.makeImage", the memory stay down. Reading that func doc, it says that it copies the data from the context. So I'm thinking that there is some data that get copied and one copy is not released.

The 'problem' is that Swift automatically handle the CoreFoundations memory retains/release, so I have no way to handle it.

As you can see I tried with an autoreleasepool, but it's not working.

Any idea?

Thanks

func image(from sampleBuffer:CMSampleBuffer) -> UIImage?{

    /* https://developer.apple.com/library/content/qa/qa1702/_index.html */
    let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

    guard imageBuffer != nil else{
        return nil
    }

    return autoreleasepool{ () -> UIImage in

        CVPixelBufferLockBaseAddress(imageBuffer!, .readOnly);

        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer!)

        // Get the number of bytes per row for the pixel buffer
        let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!);
        // Get the pixel buffer width and height
        let width = CVPixelBufferGetWidth(imageBuffer!);
        let height = CVPixelBufferGetHeight(imageBuffer!);

        let colorSpace = CGColorSpaceCreateDeviceRGB();

        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)
            .union(.byteOrder32Little)

        // Create a bitmap graphics context with the sample buffer data
        let context = CGContext(data: baseAddress, width: width, height: height, bitsPerComponent: 8,
                                bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue);

        // Create a Quartz image from the pixel data in the bitmap graphics context
        let quartzImage = context!.makeImage();
        // Unlock the pixel buffer
        CVPixelBufferUnlockBaseAddress(imageBuffer!,.readOnly);

        let image = UIImage(cgImage: quartzImage!)

        return image
    }
}

Answer 1

I found the solution to my problem.

I didn't know that UIKit is not thread safe. It appears that executing that code in a thread different from the main, does not allow the system to free the memory as it should.

On the main thread, the memory is correctly released and managed.

Answer 2

Like other Core* libraries which are not ARC compliant , you probably need to wrap the CM call inside the autorelease.

func image(from sampleBuffer:CMSampleBuffer) -> UIImage?{
 

    var ret: UIImage?
    autoreleasepool {

        /* https://developer.apple.com/library/content/qa/qa1702/_index.html */
        let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)


        CVPixelBufferLockBaseAddress(imageBuffer!, .readOnly);

        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer!)

        // Get the number of bytes per row for the pixel buffer
        let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!);
        // Get the pixel buffer width and height
        let width = CVPixelBufferGetWidth(imageBuffer!);
        let height = CVPixelBufferGetHeight(imageBuffer!);

        let colorSpace = CGColorSpaceCreateDeviceRGB();

        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)
            .union(.byteOrder32Little)

        // Create a bitmap graphics context with the sample buffer data
        let context = CGContext(data: baseAddress, width: width, height: height, bitsPerComponent: 8,
                                bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue);

        // Create a Quartz image from the pixel data in the bitmap graphics context
        let quartzImage = context!.makeImage();
        // Unlock the pixel buffer
        CVPixelBufferUnlockBaseAddress(imageBuffer!,.readOnly);

        let image = UIImage(cgImage: quartzImage!)

        ret = image
    }
    return ret
}

See this answer for details.