如何使用OpenCV在Android中将原始图像转换为RGB？

Question

I need to make some processing on each raw frame and displaying it in the preview. Here's the code I'm using (in Kotlin):

fun Image.toLinearRgb(): Mat {
    val bayer16Bit = Mat(height, width, CvType.CV_16UC1, planes[0].buffer)
    val bayer8Bit = Mat().apply {
        bayer16Bit.convertTo(this, CvType.CV_8UC1, 0.0625)
    }
    val rgb8Bit = Mat().apply {
        Imgproc.cvtColor(bayer8Bit, this, Imgproc.COLOR_BayerGR2RGB)
    }
    bayer16Bit.release()
    bayer8Bit.release()
    return rgb8Bit
}

I've ended up getting a strange rgb frame with a lot of green pixels.

According to the docs, the raw image (through ImageFormat.RAW_SENSOR) is a 16 bit, single channel, bayered one. I'm probably missing something because just allocating the byte buffer in a CV_16UC1 mat and converting it to RGB don't work. Also, it seems that the resulting frame is too dark even for a linear color space. I'd appreciate any help!

Answer 1

There are different bayer pattern layouts, and without knowing the camera's layout, I'd guess you got the wrong one. Opencv refers to them by the first 2 colors in the upper left hand corner: cv::COLOR_Bayer BG 2BGR

In opencv, these are supported by using different flags:

cv::COLOR_BayerRG2BGR
cv::COLOR_BayerGR2BGR
cv::COLOR_BayerGB2RGB

...etc. A full list can be found here: https://docs.opencv.org/3.1.0/de/d25/imgproc_color_conversions.html

You can try to research your platform's bayer layout, or you can just experiment till it works. Good Luck!