LibGDX detecting the mouse click on mesh triangles

Question

I have been trying to add mouse click detection on the triangles of the mesh, but it seems that I am doing something wrong and I cannot figure out how to solve the problem.

So before explaining the problem I will define the environment(the full code is available at http://pastebin.com/TxfNuYXZ ):

Camera position

cam = new OrthographicCamera(10, 9);
cam.position.set(0, 5.35f, 2f);
cam.lookAt(0, 0, 0);
cam.near = 0.5f;
cam.far = 12f;

Mesh renders 4 vertices.

mesh = new Mesh(true, NUM_COLUMNS * NUM_LINES, (NUM_COLUMNS * 6 - 6) * (NUM_LINES - 1), VertexAttribute.Position(), VertexAttribute.ColorUnpacked());
mesh.setVertices(new float[] { 
    0, 0, 0, 0, 1, 0, 1,
    1, 0, 0, 0, 1, 0, 1, 
    0, 0, 1, 0, 1, 0, 1, 
    1, 0, 1, 0, 1, 0, 1 });
mesh.setIndices(new short[] { 2, 0, 1, 2, 3, 1 });

So when I run the application I try to check if the click was done inside some of the triangles of the mesh. Now the result depends on the position of the camera. When the camera has almost top view(like in the following picture), corresponding to around 6 on Y axes, the click point is being correctly translated to the coordinates and corresponds to what is actually being seen.

When I move camera on the Y axes to lower position (around 2 or 3), so the image looks like the following one the click is being detected in the completely wrong positions (the red line shows the place where the click is detected).. Which seems to be right according to the coordinates, but not according to what is being seen..

I would like to understand what an I missing to be able to detect clicks on what actually is being seen? The code I use to detect the click is the following:

@Override
public boolean touchDown(int screenX, int screenY, int pointer, int button) {
    Ray ray = cam.getPickRay(screenX, screenY);
    Vector3 intersection = new Vector3();
    float[] v = new float[NUM_COLUMNS * NUM_LINES * VERTEX_SIZE];
    short[] i = new short[(NUM_COLUMNS * 6 - 6) * (NUM_LINES - 1)];
    mesh.getIndices(i);
    if (Intersector.intersectRayTriangles(ray, mesh.getVertices(v), i, VERTEX_SIZE, intersection)) {
        System.out.println(intersection);
    }
    return false;
}

Thanks a lot for your help!

Answer 1

Basically, after several days of drawing and some math I have found the source of the problem. The Vertex Shader, in order to determine the position of the vertices, was performing a_position * u_projectionViewMatrix multiplication, which was resulting on what was looking fine on the screen, but actually when you compare with actual coordinates of the mesh it was wrong. Now if you check the examples at enter link description here , you can see, that gl_Position is being calculated by multiplying u_projectionViewMatrix * a_position . Making the correct calculation made the trick.

I also had to change the camera to perspective, since the mesh was not rendered how I wanted it to.