Calculate 3D rotation of a plane given 3 points and their relative position

Question

I'm trying to rotate a plane in 3D in Unity, given three points. These points are not always the same, but they are always on the red dots in this image

I know the absolute positions of these points, along with their position relative to the plane. I might for example know that point (-5, 5) is located on (10, -4, 13).

I know that when the three points are on a line (eg (-5, 5), (0, 5) and (5, 5)) it's not possible to calculate the complete rotation, so I already throw an exception when this is the case. However, when the three points are not on a single line, it should be possible to calculate the complete rotation.

So far, I've used the code below to rotate, but this misses the rotation around the y-axis (the example uses point (-5, 5), (5, 5) and (-5, -5).

Vector3 p1 = Point1.transform.position;// point 1 absolute position
Vector3 p1Relative = new Vector3(-5, 0, 5);
Vector3 p2 = Point2.transform.position;// point 2 absolute position
Vector3 p2Relative = new Vector3(5, 0, 5);
Vector3 p3 = Point3.transform.position;// point 3 absolute position
Vector3 p3Relative = new Vector3(-5, 0, -5);

Gizmos.DrawSphere(p1, .1f);
Gizmos.DrawSphere(p2, .1f);
Gizmos.DrawSphere(p3, .1f);

Vector3 normal = Vector3.Cross(p2 - p1, p3- p1);
rotator.transform.up = normal;

How would I expand or change this code to include the rotation around the Y-axis? Thank you in advance.

Answer 1

It sounds like you actually have the given points

• Upper Left corner
• Upper Right corner
• Lower Left corner

So you could probably get both directions and simply assume that the given 3 points actually are what they claim to be

var ul = UpperLeft.transform.position;
var ur = UpperRight.transform.position;
var ll = LowerLeft.transform.position;

// actually ignored since setting right and up is already enough
// need it though to calculate the up vector and the final position
var forward = ul - ll;
var right = ur - ul;

var up = Vector3.Cross(forward, right);
rotator.transform.up = up;
rotator.transform.right = right;

// and finally if needed place the center 
rotator.transform.position = ul + right * 0.5f - forward * 0.5f;

Answer 2

Once you figure out the forward and up directions, Quaternion.LookRotation makes this easy:

struct VirtualPoint 
{
    Vector3 absolutePos;
    Vector3 relativePos;
}

// Uses list of points, but works for 3
void GetMinMaxPoints(List<VirtualPoint> points, out VirtualPoint xMinP, 
        out VirtualPoint xMaxP, out VirtualPoint yMinP, 
        out VirtualPoint yMaxP)
{
    Debug.Assert(points.Count >= 1);

    xMinP = xMaxP = yMinP = yMaxP = points[0];

    float xMin, xMax, yMin, yMax;
    xMin = xMax = points[0].relativePos.x;
    yMin = yMax = points[0].relativePos.y;

    foreach (Virtual point: points) 
    {
        Vector3 pos = point.relativePos;
        if (pos.x < xMin)
        {
            xMin = pos.x;
            xMinP = point;
        }
        else if (pos.x > xMax)
        {
            xMax = pos.x;
            xMaxP = point
        }

        if (pos.y < yMin) 
        {
            yMin = pos.y;
            yMinP = point;
        }
        else if (pos.y > yMax)
        {
            yMax = pos.y;
            yMaxP = point;
        }
    }
}

Quaternion GetPlaneRotation(List<VirtualPoint> points)
{
    Debug.Assert(points.Count >= 3);

    GetMinMaxPoints( out VirtualPoint xMinP, out VirtualPoint xMaxP,
        out VirtualPoint yMinP, out VirtualPoint yMaxP);
    
    if (xMinP.absolutePos == xMaxP.absolutePos 
            || yMinP.absolutePos == xMinP.absolutePos)
    {
        throw new System.Exception("points don't define plane");
    }

    Vector3 forwardDir = yMaxP.absolutePos - yMinP.absolutePos;
    Vector3 rightdir = xMaxP.absolutePos - xMaxP.absolutePos;

    Vector3 upDir = Vector3.Cross(forwardDir, rightDir);

    return Quaternion.LookRotation(forwardDir, upDir);
}