YUV image processing

Question

I'm developing an android app that for every YUV image passed from camera, it randomly pick 10 pixels from it and check if they are red or blue. I know how to do this for RGB images, but not for YUV format. I cannot convert it pixel by pixel into a RGB image because of the run time constrains.

Answer 1

I'm assuming you're using the Camera API's preview callbacks, where you get a byte[] array of data for each frame.

First, you need to select which YUV format you want to use. NV21 is required to be supported, and YV12 is required since Android 3.0. None of the other formats are guaranteed to be available. So NV21 is the safest choice, and also the default.

Both of these formats are YUV 4:2:0 formats; the color information is subsampled by 2x in both dimensions, and the layout of the image data is fairly different from the standard interleaved RGB format. FourCC.org's NV21 description , as one source, has the layout information you want - first the Y plane, then the UV data interleaved. But since the two color planes are only 1/4 of the size of the Y plane, you'll have to decide how you want to upsample them - the simplest is nearest neighbor. So if you want pixel (x,y) from the image of size (w, h), the nearest neighbor approach is:

Y = image[ y * w + x];
U = image[ w * h + floor(y/2) * (w/2) + floor(x/2) + 1]
V = image[ w * h + floor(y/2) * (w/2) + floor(x/2) + 0]

More sophisticated upsampling (bilinear, cubic, etc) for the chroma channels can be used as well, but what's suitable depends on the application.

Once you have the YUV pixel, you'll need to interpret it. If you're more comfortable operating in RGB, you can use these JPEG conversion equations at Wikipedia to get the RGB values.

Or, you can just use large positive values of V (Cr) to indicate red, especially if U (Cb) is small.

Answer 2

From the answer of Reuben Scratton in Converting YUV->RGB(Image processing)->YUV during onPreviewFrame in android?

You can make the camera preview use RGB format instead of YUV.

Try this:

Camera.Parameters.setPreviewFormat(ImageFormat.RGB_565);

Answer 3

YUV is just another colour space. You can define red in YUV space just as you can in RGB space. A simple calculator suggests an RGB value of 255,0,0 (red) should appear as something like 76,84,255 in YUV space so just look for something close to that.