How Do I UnProject Screen Coordinates in OpenGL ES 2.0 (Android)?

Question

I have been working for the last few days to "unproject" our app's touch events into our Renderer's coordinate space. In the pursuit of this goal, I have tried various custom unproject methods and alternative techniques (including trying to convert the coordinates using the scaling factor & transform values) to no end. I have gotten close (where my touches are slightly off) however my attempts using GLU.gluUnProject have been way off, usually placing the coordinates around the center of the view. The "closest" results were produced by Xandy's method however even these are usually off. My primary questions are how do I setup my viewport matrix and am I passing GLU.gluUnProject correct parameters? My math is based on the answer to this question . Here are the relevant excerpts of my code (showing how I setup my matrices and my current attempt):

    public void onSurfaceChanged(GL10 gl, int width, int height) {
            // Set the OpenGL viewport to fill the entire surface.
            glViewport(0, 0, width, height);
            ...
            float ratio = (float) width / height;
            Matrix.frustumM(mProjectionMatrix, 0, -ratio, ratio, -1, 1, 3, 7);
    }

    public void onDrawFrame(GL10 gl) {
            glClear(GL_COLOR_BUFFER_BIT);

            Matrix.setLookAtM(mViewMatrix, 0, 0f, 0f, -4.5f, 0f, 0f, 0f, 0f, 1f, 0f);
            Matrix.scaleM(mViewMatrix, 0, mScaleFactor, mScaleFactor, 1.0f);
            Matrix.translateM(mViewMatrix, 0, -mOffset.x, mOffset.y, 0.0f);

            Matrix.multiplyMM(mModelMatrix, 0, mProjectionMatrix, 0, mViewMatrix, 0);
            ...
    }

    public PointF convertScreenCoords(float x, float y) {
            int[] viewPortMatrix = new int[]{0, 0, (int)mViewportWidth, (int)mViewportHeight};
            float[] outputNear = new float[4];
            float[] outputFar = new float[4];

            y = mViewportHeight - y;
            int successNear = GLU.gluUnProject(x, y, 0, mModelMatrix, 0, mProjectionMatrix, 0, viewPortMatrix, 0, outputNear, 0);
            int successFar = GLU.gluUnProject(x, y, 1, mModelMatrix, 0, mProjectionMatrix, 0, viewPortMatrix, 0, outputFar, 0);

            if (successNear == GL_FALSE || successFar == GL_FALSE) {
                throw new RuntimeException("Cannot invert matrices!");
            }

            convert4DCoords(outputNear);
            convert4DCoords(outputFar);

            float distance = outputNear[2] / (outputFar[2] - outputNear[2]);
            float normalizedX = (outputNear[0] + (outputFar[0] - outputNear[0]) * distance);
            float normalizedY = (outputNear[1] + (outputFar[1] - outputNear[1]) * distance);

            return new PointF(normalizedX, normalizedY);
    }

convert4DCoords is simply a helper function that divides each coordinate (x, y, z) of an array by w. mOffset and mScaleFactor are the translation and scaling parameters (given by a ScaleGestureDetector within our GLSurfaceView )

Based on everything I have read this should be working however it is consistently wrong and I am not sure what else to try. Any help/feedback would be greatly appreciated!

Answer 1

I didn't look through all of your math, but some of your matrix operations and specifications look wrong to me. That is most likely at least part of your problem.

Looking at what you call mViewMatrix first:

Matrix.setLookAtM(mViewMatrix, 0, 0f, 0f, -4.5f, 0f, 0f, 0f, 0f, 1f, 0f);
Matrix.scaleM(mViewMatrix, 0, mScaleFactor, mScaleFactor, 1.0f);
Matrix.translateM(mViewMatrix, 0, -mOffset.x, mOffset.y, 0.0f);

There's nothing necessarily wrong with this. But most people would probably only call the first part, which is the result of setLookAtM() , a View matrix. The rest looks then more like a Model matrix. But since rendering normally only uses the product of View and Model matrix, this is just terminology. It would seem fair to say that what you store in mViewMatrix is your ModelView matrix.

Then the naming gets stranger here:

Matrix.multiplyMM(mModelMatrix, 0, mProjectionMatrix, 0, mViewMatrix, 0);

This calculates the product of Projection and View matrix. So this would be a ViewProjection matrix. Storing this in a variable called mModelMatrix is confusing, and I think you might have set yourself a trap with this naming.

This leads us to the glUnproject() call:

GLU.gluUnProject(x, y, 0, mModelMatrix, 0, mProjectionMatrix, 0, viewPortMatrix, 0, outputNear, 0);

Based on the above, what you're passing for the two matrix arguments (the viewport is not really a matrix) are the ViewProjection and the Projection matrix. Where based on the method definition, you should be passing the ModelView and the Projection matrix.

Since we established that what you call mViewMatrix corresponds to a ModelView matrix, this should start working much better if you change just that argument:

GLU.gluUnProject(x, y, 0, mViewMatrix, 0, mProjectionMatrix, 0, viewPortMatrix, 0, outputNear, 0);

Then you can get rid of the mModelMatrix that is not a Model matrix at all.