Color picking on android - glReadPixels rounding errors

Question

I am using color picking in opengl es on android and i am calculating a color key to compare it to the values i get from glReadPixels:

ByteBuffer PixelBuffer = ByteBuffer.allocateDirect(4);
PixelBuffer.order(ByteOrder.nativeOrder());
gl.glReadPixels(x, y, 1, 1, GL10.GL_RGBA, GL10.GL_UNSIGNED_BYTE, PixelBuffer);
byte b[] = new byte[4];
PixelBuffer.get(b);
String key = "" + b[0] + b[1] + b[2];

This key can be calculated manually for any color with:

public static byte floatToByteValue(float f) {
    return (byte) ((int) (f * 255f));
}

First the float value is converted to an intvalue and then castet to byte. The float values describe the colorchanels red green blue (from 0.0f to 1.0f). Example: 0.0f is converted to 255 (now integer) and then from 255 to -1 in byte

This works fine but opengl seems to make rounding errors some times. Example:

0.895 -> -28  and opengl returns -27
0.897 -> -28  and opengl returns -27
0.898 -> -28  and opengl returns -27
0.8985 -> -27 and opengl returns -27
0.899 -> -27  and opengl returns -26
0.9 -> -27    and opengl returns -26
0.91 -> -24   and opengl returns -24

maybe my calculation method isnt correct? Does anyone have an idea how to avoid these deviations?

Answer 1

for example if you set red float(31 / 255), I think the transformation is like this.

31 (0001_1111) transforms to 3(0000_0011) if the color format is RGB565(default)

then we use glReadPixels() to get the value

3(0000_0011) transforms to 24(0001_1000)

In a word, if you set 31 for red color you will get 24 in the end.

The key to kill the rounding error is the method to transform from RGB565 to RGB88 which you did not do.

You can have a try. Good luck.

Answer 2

Java bytes are signed.

-28 = 0xe4 in signed byte = 228 decimal if you treat the 0xe4 as unsigned.
228/255 = 0.894 in float