How to check if a point is inside a quad in perspective projection?

Question

I want to test if any given point in the world is on a quad/plane? The quad/plane can be translated/rotated/scaled by any values but it still should be able to detect if the given point is on it. I also need to get the location where the point should have been, if the quad was not applied any rotation/scale/translation.

For example, consider a quad at 0, 0, 0 with size 100x100, rotated at an angle of 45 degrees along z axis. If my mouse location in the world is at ( x, y, 0, ), I need to know if that point falls on that quad in its current transformation? If yes, then I need to know if no transformations were applied to the quad, where that point would have been on it? Any code sample would be of great help

Answer 1

A ray-casting approach is probably simplest:

1. Use gluUnProject() to get the world-space direction of the ray to cast into the scene. The ray's origin is the camera position.
2. Put this ray into object space by transforming it by the inverse of your rectangle's transform. Note that you need to transform both the ray's origin point and direction vector.
3. Compute the intersection point between this ray and the XY plane with a standard ray-plane intersection test.
4. Check that the intersection point's x and y values are within your rectangle's bounds, if they are then that's your desired result.

A math library such as GLM will be very helpful if you aren't confident about some of the math involved here, it has corresponding functions such as glm::unProject() as well as functions to invert matrices and do all the other transformations you'd need.