What does the PBO buffer content mean?

Question

I was trying to implement a color picking system by using a PBO(pixel buffer object using OpenGL), and when I finished, I realized the numbers that came out of the PBO when mapped didn't make any sense at all. I made my application render big squares of different colors and the result DID change between these different colors, but even after analyzing the numbers, I can't make sense of it.

For example, clicking on pure red gave me bytes of (-1,0,0), while pure blue gave (0,0,-1), but against all logic, pure green gives (-1,0,-1), cyan also gives (-1,0,0), and yellow gives (76,-1,0).

Obviously these numbers are wrong, given that two different colors can result in the same byte formation. Shouldn't a fully red color be (127,0,0)?

Here is the code I used for initialization, size 3 because I am only reading one pixel.

pboid = glGenBuffersARB(); //Initialize buffer for pbo
glBindBufferARB(GL_PIXEL_PACK_BUFFER_EXT, pboid); //bind buffer
glBufferDataARB(GL_PIXEL_PACK_BUFFER_EXT, 3,GL_DYNAMIC_READ); //create a pbo with 3 slots
glBindBufferARB(GL_PIXEL_PACK_BUFFER_EXT, 0); //unbind buffer

And here is the code I used for reading the pixels

glBindBufferARB(GL_PIXEL_PACK_BUFFER_EXT, pboid); //Bind the pbo
glReadPixels((int)lastMousePosition.x,(int)lastMousePosition.y,1,1, GL_RGB, GL_UNSIGNED_BYTE, 0); //Read 1 pixel
ByteBuffer colorBuffer = glMapBufferARB(GL_PIXEL_PACK_BUFFER_EXT, GL_READ_ONLY_ARB); //Map the buffer so we can read it
for(int x = 0; x < colorBuffer.limit(); x++)
{
    System.out.println("color byte: " + colorBuffer.get(x)); //Print out the color byte
}
glUnmapBufferARB(GL_PIXEL_PACK_BUFFER_EXT); //Unmap the buffer so it can be used again
glBindBufferARB(GL_PIXEL_PACK_BUFFER_EXT, 0); //Unbind the pbo

If I am wrong in any assumptions I have made, please correct me. I am planning perhaps to use this system to tell which gui element is being clicked by rendering each of them to an fbo with a unique color and testing which pixel color was clicked on. Thanks in advance for anyone who can help!

Answer 1

At long last I have finally found the issue!

Firstly, using Byte.toUnsignedInt(byte), you can turn the color that the pbo gives you into your traditional 0-255 range rgb numbers.

Secondly, this being the primary issue, when OpenGL asks for pixel coordinates to fill a pbo, it is relative to bottom right. My issue was that I was using GLFW which gives coordinates relative to top right, meaning color picking in the vertical middle of the screen was accurate, but that it was getting the inverse part of the screen I was looking for when color picking elsewhere. To fix this, simply subtract the mouse click's y coordinate from the window height.

Thanks for the help and ideas!

Answer 2

There are a couple of possibilities, but I don't have my openGL system set up to test - but you can try these anyhow. Also I don't know Java too well ( C,C++ etc is my domain)

EDIT

1) You have asked for GL_UNSIGNED_BYTE data from glReadPixels() , but you are printing out in signed format. GL_UNSIGNED_BYTE has values 0-256, so negative values are not possible! Try to format you printout for UNSIGNED_BYTE and see where that leads. (from your code I can see this is now fixed).

2) As derhass pointed out in his comments, you should not be using the ARB (architecture Review Board) extension versions of OpenGL buffer functions since these are part of OpenGL core for quite a long time now. See https://www.khronos.org/opengl/wiki/History_of_OpenGL for version history. From this I can see glBindBufferARB (for example) was deprecated in 2003. It may or not impact your particular problem, but replace glXXXXXARB() with glXXXXX() thorughout, and make sure your OpenGL libraries are recent (v4 or later).

3) Also credit derhass, and reading your GitHub code, your getMousePosition() via glfwGetCursorPos returns screen coordinates (0,0 is top left of your window) so you need to convert to viewport coordinates (0,0 is bottom left) to read the framebuffer. Your code at GitHub seems not to be making the conversion.

4) Also credit derhass, you dont' need to use PBO at all for basic color picking. glReadPixels() default target is the framebuffer, so you can safely dispense with the VBO and get color, depth and stencil data directly from the framebuffer. (you need to enable the depth and stencil buffers).

5) If you are selecting on a 3D scene, you will also need to convert (unproject) the viewport coordinates and depth back to worldcoordinates to be able to identify which object you have clicked on. See https://en.wikibooks.org/wiki/OpenGL_Programming/Object_selection for some ideas on selection.

I hope all this helps a bit, although it feels a like learning experience for both of us.