Android OpenGL ES: How do you select a 2D object?

Question

I have been searching for a introductory to 2D selection in OpenGL ES in Stack Overflow. I mostly see questions about 3D.

I'm designing a 2D tile-based level editor on Android 4.0.3, using OpenGL ES. In the level editor, there is a 2D, yellow, square object placed in the center of the screen. All I wanted is to detect to see if the object has been touched by a user.

In the level editor, there aren't any tiles overlapping. Instead, they are placed side-by-side, just like two nearby pixels in a bitmap image in MS Paint. My purpose is to individually detect a touch event for each square object in the level editor.

The object is created with a simple vertex array, and using GL_TRIANGLES to draw 2 flat right triangles. There are no manipulations and no loading from a file or anything. The only thing I know is that if a user touches any one of the yellow triangles, then both yellow triangles are to be selected.

Could anyone provide a hint as to how I need to do this? Thanks in advance.

EDIT:

This is the draw() function:

public void draw(GL10 gl) {
    gl.glPushMatrix();
    gl.glTranslatef(-(deltaX - translateX), (deltaY - translateY), 1f);
    gl.glColor4f(1f, 1f, 0f, 1f);
    //TODO: Move ClientState and MatrixStack outside of draw().
    gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
    gl.glVertexPointer(2, GL10.GL_FLOAT, 0, vertices);
    gl.glDrawArrays(GL10.GL_TRIANGLES, 0, 6);
    gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
    gl.glPopMatrix();
}

EDIT 2:

I'm still missing some info. Are you using a camera? or pushing other matrixes before the model rendering?. For example, if you are using an orthographic camera, you can easily unproject your screen coordinates [x_screen, y_screen] like this (y is analogous):

I'm not using a camera, but I'm probably using an orthographic projection. Again, I do not know, as I'm just using a common OpenGL function. I do pushing and popping matrices, because I plan on integrating many tiles (square 2D objects) with different translation matrices. No two tiles will have the same translation matrix M.

Is a perspective projection the same as orthographic projection when it comes to 2D? I do not see any differences between the two.

Here's the initial setup when the surface is created (a class extending GLSurfaceView , and implementing GLSurfaceView.Renderer ):

public void onSurfaceChanged(GL10 gl, int width, int height) {
    gl.glViewport(0, 0, width, height);
}

public void onSurfaceCreated(GL10 gl, EGLConfig arg1) {
    reset();
}
public void onDrawFrame(GL10 gl) {
    clearScreen(gl);
    gl.glMatrixMode(GL10.GL_PROJECTION);
    gl.glLoadIdentity();
    gl.glOrthof(0f, super.getWidth(), 0f, super.getHeight(), 1, -1);
    gl.glMatrixMode(GL10.GL_MODELVIEW);
    gl.glLoadIdentity();
    canvas.draw(gl);
}

private void clearScreen(GL10 gl) {
    gl.glClearColor(0.5f, 1f, 1f, 1f);
    gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
}

Answer 1

A basic approach would be the following:

1. Define a bounding box for each "touchable" object. This could be just a rectangle (x, y, width, height).
2. When you update a tile in the world you update its bounding box (completely in world coordinates).
3. When user touches the screen, you have to unproject screen coordinates to world coordinates
4. Check if unprojected point overlaps with any bounding box.

Some hints on prev items. [Edited]

• 1 and 2. You should have to keep track of where you are rendering your tiles. Store their position and size. A rectangle is a convenient structure. In your example it could be computed like this. And you have to recompute it when model changes. Lets call it Rectangle r:

   rx = yourTile.position.x -(deltaX - translateX) ry = yourTile.position.y -(deltaY - translateY) r.width= yourTile.width //as there is no model scaling r.height = yourTile.height// 

• 3 - if you are using an orthographic camera, you can easily unproject your screen coordinates [x_screen, y_screen] like this (y is analogous):

   x_model = ((x_screen/GL_viewport_width) -0.5 )*camera.WIDTH + Camera.position.x 

• 4 - For each of your Rectangles check if [x_model; y_model] is inside it.

[2nd Edit] By the way you are updating your matrixes, you can consider you are using a camera with postition surfaceView.width()/2, surfaceView.height()/2. You are matching 1 pixel on screen to 1 unit in world, so you dont need to unproject anything. You can replace that values on my formula and get x_screen = x_model - (You 'll need to flip the Y component of the touch event because of the Y grows downwards in Java, and upwards in GL).

Final words. If user touches point [x,y] check if [x, screenHeight-y]* hits some of your rectangles and you are done. Do some debugging, log the touching points and see if they are as expected. Generate your rectangles and see if they match what you see on screen, then is a matter of checking if a point is inside a rectangle.

I must tell you that you should not set the camera to screen dimensions, because your app will look dramatically different on different devices. This is a topic on its own so i won't go any further, but consider defining your model in terms of world units - independent from screen size. This is getting so off-topic, but i hope you have gotten a good glimpse of what you need to know!

*The flipping i told you. PS: stick with the orthographic projection (perspective would be more complex to use).

Answer 2

Please, allow me to post a second answer to your question. This is completely more high-level/philosophical. May be a silly, useless answer but, I hope it will help someone new to computer graphics to change it's mind to "graphics mode".

You can't really select a triangle on the screen. That square is not 2 triangles. That square is just a bunch of yellow pixels. OpenGL takes some vertices, connects them, process them and colors some pixels on the screen. At one stage on the graphics pipeline even geometrical information is lost, and you only have isolated pixels. That's analogous to a letter printed by a printer on a paper. You usually don't process information from a paper (ok, maybe a barcode reader does :D)

If you need to further process your drawings, you have to model them and process them yourself with auxiliary data structures. That's why I suggested you created a rectangle to model your tiles. You create your imaginary "world" of objects, and then render them to screen. The user touch-event does not belong to the same world, so you have to "translate" screen coordinates into your world coordinates. Then you change something in your world (may be the user drags her finger and you have to move an object), and back again tell OpenGL to render your world to screen.

You should operate on your model, not the view. Meshes are more of a view thing, so you shouldn't mix them with the model information, it's a good practice to separate both things. (please, an expert correct me, I'm quite a graphics hobbyist)

Answer 3

Have you checked out LibGDX ?

Makes life so much easier when working with OpenGL ES.