Unprojecting 2D screen coordinates to 3D world coordinates

Question

I'm doing some projecting and after projecting to 2D I want to the reverse operation, unprojecting to 3D. The only step I am confused at is when going from NDC to HDC, more specific how to do reverse of the perspective division? Currently when doing the reverse I end up with a -1 w-value which isn't going to do much when dividing or multiplying, or maybe there is something else I am overseeing. This is the code I currently have:

    private Vector3f projectTo3D(Matrix4f projection, Matrix4f view, Matrix4f world, Vector3f vertex) {

    Matrix4f.invert(projection, projection);
    Matrix4f.invert(view, view);
    Matrix4f.invert(world, world);

    System.out.println("projectTo3D - Screen Coordinate: " + vertex.toString());

    ....

    System.out.println("NDC: " + x + ", " + y + ", " + z);
    final Vector4f HDC = new Vector4f(x, y, z, -1f);
    HDC.x = HDC.x / HDC.w;
    HDC.y = HDC.y / HDC.w;
    HDC.z = HDC.z / HDC.w;

    System.out.println("HDC before transform: " + HDC.toString());
    Matrix4f.transform(worldProjectionView, HDC, HDC);
    System.out.println("HDC after transform: " + HDC.toString());

    final Vector3f worldCoordinates = new Vector3f(HDC.x, HDC.y, HDC.z);
    System.out.println("World coordinate: " + worldCoordinates.toString());
    return worldCoordinates;
}

---

projectTo2D - World Coordinate: Vector3f[-50.0, -50.0, 0.0]

HDC: Vector4f: -195.05373 -44.30973 -529.6105 -529.6105

NDC: Vector3f[0.3682966, 0.08366475, 0.0]

Screen coordinate: 547.31866, 325.09943, 0.0

---

projectTo3D - Screen Coordinate: Vector3f[ 547.31866, 325.09943, 0.0 ]

NDC: 0.36829662, 0.083664775, -1.0

HDC before transform: Vector4f: -0.36829662 -0.083664775 1.0 -1.0

HDC after transform: Vector4f: -292.38623 405.26837 -291.50772 1.0

World coordinate: Vector3f[-292.38623, 405.26837, -291.50772]

I am following this diagram:

Answer 1

You actually have a linear algebra problem here, and one which is unfortunately unsolvable. When you project a 3D-coordinate space onto a 2D space, which is the job of the model->viewport transform, you're losing a dimension along the way, and the reverse problem becomes under-constrained.

What I mean by that is, if you only have a vector in a 2D space (such as a point in a screen), there is an infinite number of 3D vectors which could project onto the same 2D vector, forming a new 1D vector space. And since the projection is not an affine (linear) operation, this means your 1D space is not a straight line; rather, a curved line in 3D space.

Answer 2

I seem to have fixed this myself. My mistake was setting the near plane of the projection matrix to 0 and this should be minimum 0.1