尝试使用 openGL 从屏幕上的 2d 单击中获取 3d 点

Question

i have a PyOpenGL project. i render a stadium and some pcd clouds. and i want to extract a point from the screen. i have the X,Y values and i am trying to use gluUnProject or glReadPixelsf to get the point in x,y,z.

but i am getting weird numbers for example: i ZOOMED as much as i can to the 0,0,0 point with my camera and used the

 worldCoordinate1 = gluUnProject(x, realy, 0.0, mvmatrix, projmatrix, vport)

gets :

(-79.71705354658924, 3.024572101740394, -1.8905707759570176)

which looks fine. this is truly the camera location. but my problem starts with the far plane.

worldCoordinate2 = gluUnProject(x, realy, 1.0, mvmatrix, projmatrix, vport)

ideally i want this to get me the point colliding with any buffer object drawn. but i am getting

  (2098.1759117202328, -121.52711517535181, 77.12367509703613)

my function looks like this:

def get_clicked_location(x, y):
    vport = glGetIntegerv(GL_VIEWPORT)
    mvmatrix = glGetDoublev(GL_MODELVIEW_MATRIX)
    projmatrix = glGetDoublev(GL_PROJECTION_MATRIX)
    realy = vport[3] - y
    worldCoordinate1 = gluUnProject(x, realy, 0.0, mvmatrix, projmatrix, vport)
    worldCoordinate2 = gluUnProject(x, realy, 1.0, mvmatrix, projmatrix, vport)
    print worldCoordinate1
    print worldCoordinate2
    data = glReadPixelsf(x, realy, 1, 1, GL_RGB, GL_UNSIGNED_BYTE)
    pick = map(ord, data)
    print pick

what i want is to get the x,y,z point colliding with the buffer drawn on screen. but i cant seem to find the way. also i found some people using glreadpixelsf but i don't think i understand how to combine them thanks for the help.

Answer 1

You can use glReadPixels to read the depth from the depth buffer of the default framebuffer. Use the format GL_DEPTH_COMPONENT to do so:

depth = glReadPixels(x, realy, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT)

This depth can be used with gluUnProject , to find the world (or model) coordinate of the fragment:

vport      = glGetIntegerv(GL_VIEWPORT)
mvmatrix   = glGetDoublev(GL_MODELVIEW_MATRIX)
projmatrix = glGetDoublev(GL_PROJECTION_MATRIX)

worldCoordinate1 = gluUnProject(x, realy, depth, mvmatrix, projmatrix, vport)

See also get 3d point in space using 2d point in image in python opengl