How to scale a rotated rectangle in canvas

Question

What is the correct math formula to use to scale a rectangle by corner handles when the shape is rotated?

UPDATED:

The question is more about the math behind the mouse down event on the handles and the actual sizing of the shape. What is to proper math used to calculate the shape position and scaled size when the handle is being moved on a rotated shape?

I have created a fiddle with the project to show an example: https://jsfiddle.net/8b5zLupf/38/

The gray shape on the canvas in the fiddle can be moved and scaled but because the shape is rotated, the math to scale the shape and keep the shape position when scaling is being calculated incorrectly.

The project will scale but it does not lock the shape by the opposite point and scale uniformly.

The area of code I use to scale a shape with aspect is below:

resizeShapeWithAspect: function(currentHandle, shape, mouse)
{
    var self = this;
    var getModifyAspect = function(max, min, value)
    {
        var ratio = max / min;
        return value * ratio;
    };

    var modify = {
        width: 0,
        height: 10
    };
    var direction = null,
        objPos = shape.position,
        ratio = this.getAspect(shape.width, shape.height);
    switch (currentHandle)
    {
        case 'topleft':
            modify.width = shape.width + (objPos.x - mouse.x);
            modify.height = shape.height + (objPos.y - mouse.y);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = (modify.width - shape.width);
            var changeY = (modify.height - shape.height);
            objPos.x = mouse.x + changeX;
            objPos.y = mouse.y + changeY;
            break;
        case 'topright':
            modify.width = mouse.x - objPos.x;
            modify.height = shape.height + (objPos.y - mouse.y);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeY = (modify.height - shape.height);
            objPos.y = mouse.y + changeY;
            break;
        case 'bottomleft':
            modify.width = shape.width + (objPos.x - mouse.x);
            modify.height = mouse.y - objPos.y;

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = (modify.width - shape.width);
            objPos.x = mouse.x + changeX;
            break;
        case 'bottomright':
            modify.width = mouse.x - objPos.x;
            modify.height = mouse.y - objPos.y;

            this.scale(shape, modify);
            break;

        case 'top':
            var oldWidth = shape.width;
            modify.width = shape.width + (objPos.x + mouse.x);
            modify.height = shape.height + (objPos.y - mouse.y);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = ((shape.width - oldWidth) / 2);
            var changeY = (modify.height - shape.height);
            objPos.x -= changeX;
            objPos.y = mouse.y + changeY;
            break;

        case 'left':
            var oldHeight = shape.height;
            modify.width = shape.width + (objPos.x - mouse.x);
            modify.height = getModifyAspect(modify.width, shape.width, shape.height);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = (modify.width - shape.width);
            var changeY = ((shape.height - oldHeight) / 2);
            objPos.x = mouse.x + changeX;
            objPos.y -= changeY;
            break;

        case 'bottom':
            var oldWidth = shape.width;
            modify.height = mouse.y - objPos.y;
            modify.width = getModifyAspect(modify.height, shape.height, shape.width);

            this.scale(shape, modify);

            var changeX = ((shape.width - oldWidth) / 2);
            objPos.x -= changeX;
            break;

        case 'right':
            var oldHeight = shape.height;
            modify.width = mouse.x - objPos.x;
            modify.height = getModifyAspect(modify.width, shape.width, shape.height);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeY = ((shape.height - oldHeight) / 2);
            objPos.y -= changeY;
            break;
    }
}

I have modified the code above to work with angles but it does not work correctly.

resizeShapeWithAspectAndRotate: function(currentHandle, shape, mouse)
{
    var self = this;
    var getModifyAspect = function(max, min, value)
    {
        var ratio = max / min;
        return value * ratio;
    };

    var modify = {
        width: 0,
        height: 10
    };
    var direction = null,
        objPos = shape.position,
        ratio = this.getAspect(shape.width, shape.height),
        handles = shape.getHandlePositions();
    switch (currentHandle)
    {
        case 'topleft':
            var handle = this.getHandleByLabel(handles, 'topleft');
            var opositeHandle = this.getOpositeHandle(handles, 'topleft');
            var distance = canvasMath.distance(handle, mouse);

            modify.width = shape.width + (distance);
            modify.height = shape.height + (distance);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = (modify.width - shape.width);
            var changeY = (modify.height - shape.height);
            //shape.position.x = mouse.x + changeX;
            //shape.position.y = mouse.y + changeY;
            break;
        case 'topright':
            modify.width = mouse.x - objPos.x;
            modify.height = shape.height + (objPos.y - mouse.y);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeY = (modify.height - shape.height);
            objPos.y = mouse.y + changeY;
            break;
        case 'bottomleft':
            modify.width = shape.width + (objPos.x - mouse.x);
            modify.height = mouse.y - objPos.y;

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = (modify.width - shape.width);
            objPos.x = mouse.x + changeX;
            break;
        case 'bottomright':
            modify.width = mouse.x - objPos.x;
            modify.height = mouse.y - objPos.y;

            this.scale(shape, modify);
            break;

        case 'top':
            var oldWidth = shape.width;
            modify.width = shape.width + (objPos.x + mouse.x);
            modify.height = shape.height + (objPos.y - mouse.y);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = ((shape.width - oldWidth) / 2);
            var changeY = (modify.height - shape.height);
            objPos.x -= changeX;
            objPos.y = mouse.y + changeY;
            break;

        case 'left':
            var oldHeight = shape.height;
            modify.width = shape.width + (objPos.x - mouse.x);
            modify.height = getModifyAspect(modify.width, shape.width, shape.height);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeX = (modify.width - shape.width);
            var changeY = ((shape.height - oldHeight) / 2);
            objPos.x = mouse.x + changeX;
            objPos.y -= changeY;
            break;

        case 'bottom':
            var oldWidth = shape.width;
            modify.height = mouse.y - objPos.y;
            modify.width = getModifyAspect(modify.height, shape.height, shape.width);

            this.scale(shape, modify);

            var changeX = ((shape.width - oldWidth) / 2);
            objPos.x -= changeX;
            break;

        case 'right':
            var oldHeight = shape.height;
            modify.width = mouse.x - objPos.x;
            modify.height = getModifyAspect(modify.width, shape.width, shape.height);

            this.scale(shape, modify);

            /* we need to setup the shape position by getting the
            offset from where the object would have been without the
            scale and add that to the position */
            var changeY = ((shape.height - oldHeight) / 2);
            objPos.y -= changeY;
            break;
    }
},

getHandleByLabel: function(handles, label)
{
    if (handles)
    {
        for (var i = 0, maxLength = handles.length; i < maxLength; i++)
        {
            var handle = handles[i];
            if (label === handle.label)
            {
                return handle;
            }
        }
    }
    return false;
},

getOpositeHandle: function(handles)
{
    var handleLabel = this.currentHandle;
    if (handleLabel && handles)
    {
        switch (handleLabel)
        {
            case 'topleft':
                return this.getHandleByLabel(handles, 'bottomright');
            case 'top':
                return this.getHandleByLabel(handles, 'bottom');
            case 'topright':
                return this.getHandleByLabel(handles, 'bottomleft');
            case 'right':
                return this.getHandleByLabel(handles, 'left');
            case 'bottomright':
                return this.getHandleByLabel(handles, 'topleft');
            case 'bottom':
                return this.getHandleByLabel(handles, 'top');
            case 'bottomleft':
                return this.getHandleByLabel(handles, 'topright');
            case 'left':
                return this.getHandleByLabel(handles, 'right');
        }
    }
    return false;
}

Answer 1

Way to much code to go through and find a fix so here is a simple and fast way to set a scale, translation, and rotation

// scaleX, scaleY the two scales
// posX posY the position
// rotate the amount of rotation
ctx.setTransform(scaleX,0,0,scaleY,posX,posY);
ctx.rotate(rotate);

Then draw the box relative to the origin (point rotated around)

ctx.fillRect(-50,-50,100,100); /// box with center as origin
ctx.fillRect(0,0,100,100); /// box with top left as origin
ctx.fillRect(-100,-100,100,100); /// box with bottom right as origin

To restore the transform to canvas default

ctx.setTransform(1,0,0,1,0,0);

Update

Transformations, coordinates and the inverse.

To manipulate canvas objects you can use the transformation matrix. The upcoming spec lets you get the current transform and manipulate it but it is still in the experimental stage. For now you need to maintain the transform yourself.

The transformation matrix

The transformation matrix is made you of 2 vectors and a coordinate. These vectors and coordinates are always in canvas pixel coordinates and represent the direction and length of a pixels x axis, y axis and the position of the origin.

The documentation for ctx.setTransform calls the arguments a, b, c, d, e, f which obscure their actual contextual meaning. I prefer to call them xAx, xAy, yAx, yAy, ox, oy where xAx, xAy is the X Axis vector (x,y), yAx, yAy is the Y Axis vector (x,y) and ox, oy is the origin (x,y).

Thus for the default transform where a pixel is one pixel wide, one pixel in height and starts at the top right of the canvas

var xAx = 1;   // X axis vector
var xAy = 0;

var yAx = 0;   // Y axis vector
var yAy = 1;

var ox = 0;   // origin
var oy = 0;

And can be used to set the default transform (rather than use save and restore) ctx.setTransform(xAx, xAy, yAx, yAy, ox, oy);

To translate using the matrix set the origin to the canvas pixel coordinates you want.

ox = ctx.canvas.width / 2;   // centre the transformation
oy = ctx.canvas.height / 2;

To scale you just change the vector length of either the x Axis or y Axis.

var scaleX = 2;
var scaleY = 3;    

// scale x axis
xAx *= scaleX;
xAy *= scaleX;

// scale y axis
yAx *= scaleY;
yAy *= scaleY;

Rotation is a little more tricky. For now we will ignore any skew and presume that the y axis is always 0.5Pi radians (from here I will use radians in the unit of Pi. 360deg is 2R equivalent to (2 * Pi) radians) or 0.5R (90deg) from clockwise from the x axis.

To set a rotation we get the rotated unit vector for the x axis

var rotate = 1.0; // in Pi units radian

xAx = Math.cos(rotate * Math.PI); // get the rotated x axis
xAy = Math.sin(rotate * Math.PI);

yAx = Math.cos((rotate + 0.5) * Math.PI); // get the rotated y axis at 0.5R (90deg) clockwise from the x Axis
yAy = Math.sin((rotate + 0.5) * Math.PI);

We can exploit the symmetry involved to shorten the equation a little (good when you are rendering 100's to 1000's or objects). To rotate a vector 0.5R (90deg) you simply swap the x and y components negating the new x component.

// rotate a vector 0.5R (90deg)
var vx = 1;
var vy = 0;

var temp = vx;   // swap to rotate
vx = -vy;        // negate the new x
vy = temp;

// or use the ES6 destructuring syntax
[vx, vy] = [-vy, vx];  // easy as 

Thus to set up the rotation for the two axis

 // rotation now in radians
 rotate *= Math.PI; // covert from Pi unit radians to radians

 yAy = xAx = Math.cos(rotate);
 yAx = -(xAy = Math.sin(rotate));

 // shame the x of the y axis needs to be negated or ES6 syntax would be better in this case.
 [xAx, xAy] = [Math.cos(rotate), Math.sin(rotate)]; 
 [yAx, yAy] = [-xAy, xAx]; // negate the x for the y 

We can put all that together and create a matrix from its decomposed parts.

 // x, y the translation (the origin)
 // scaleX, scaleY the x and y scale,
 // r the rotation in radians 
 // returns the matrix as object
 function recomposeMatrix(x, y, scaleX, scaleY, rotate){
     var xAx,xAy,yAx,yAy;
     xAx = Math.cos(rotate);
     xAy = Math.sin(rotate);
     [yAx, yAy] = [-xAy * scaleY, xAx * scaleY];
     xAx *= scaleX;
     xAy *= scaleX;
     return {xAx, xAy, yAx, yAy, ox: x, oy :y};
 }

You can hand this matrix to the 2D context for rendering

var matrix =  recomposeMatrix(100,100,2,2,1);     
ctx.setTransform(matrix.xAx, matrix.xAy, matrix.yAx, matrix.yAy, matrix.ox, matrix.oy);

Alternative lazy programmers way

 // x, y the translation (the origin)
 // scaleX, scaleY the x and y scale,
 // r the rotation in radians 
 // returns the matrix as array
 function recomposeMatrix(x, y, scaleX, scaleY, rotate){
     var yAx,yAy;
     yAx = -Math.sin(rotate);
     yAy = Math.cos(rotate);
     return [yAy * scaleX, - yAx * scaleX, yAx * scaleY, yAy * scaleY, x, y];
 }
 var matrix = recomposeMatrix(100,100,1,1,0);
 ctx.setTransform(...matrix);

Transform a point

Now that you have the matrix you need to use it. To transform a point by a matrix you use matrix math (a lot of rules, blah blah blah) or you use vector math.

You have a point x,y and the matrix with its two axis vectors and origin. To rotate and scale you move the point alone the matrix x axis by the distance x then you move along the matrix y axis by distance y and finally you add the origin.

 var px = 100;  // point to transform
 var py = 100;

 var matrix =  recomposeMatrix(100,100,2,2,1);  // get a matrix

 var tx,ty; // the transformed point

 // move along the x axis px units     
 tx = px * matrix.xAx;
 ty = px * matrix.xAy;
 // then along the y axis py units
 tx += py * matrix.yAx;
 ty += py * matrix.yAy;
 // then add the origin
 tx += matrix.ox;
 ty += matrix.oy;

As a function

 function transformPoint(matrix,px,py){
     var x = px * matrix.xAx + py * matrix.yAx + matrix.ox;
     var y = px * matrix.xAy + py * matrix.yAy + matrix.oy;
     return {x,y};
 }

Invert the matrix.

The problem in may CG apps is locating a point relative to a rotated scaled object. We need to get a concept of coordinate systems (called spaces) being layered and separate.

For 2D this is relatively simple. You have the Screen or Canvas space that is always in pixels the matrix is [1,0,0,1,0,0] origin at 0,0, x axis 1 pixel along the top, and y axis 1 pixel down.

Then you would have world space. This is the space that rotates, scales, and translates all object in the scene. And then you have each objects local space. This is the object own separate rotation, scale and translation.

For the case of answer brevity I will ignore the Screen and World space but to say they are combined to get the eventual local space.

So we have a rotated, scaled, translated object an you want to get the coordinates relative to the object, not the screen space coordinates, but locally it its own x and y axis.

To do this you apply a transform to the screen coordinates (eg the mouse x,y) that undoes the transformation that puts the object where it is. You can get that transform by inverting the object transformation matrix.

 // mat the matrix to transform.
var rMat = {}; // the inverted matrix.
var det =  mat.xAx * mat.yAy - mat.xAy * mat.yAx; // gets the scaling factor called determinate
rMat.xAx  = mat.yAy / det;
rMat.xAy  = -mat.xAy / det;
rMat.yAx  = -mat.yAx / det;
rMat.yAy  = mat.xAx / det;
// and invert the origin by moving it along the 90deg rotated axis inversely scaled
rMat.ox = (mat.yAx * mat.oy - mat.yAy * mat.ox) / det;
rMat.oy = -(mat.xAx * mat.oy - mat.xAy * mat.ox) / det;

As a function

function invertMatrix(mat){
    var rMat = {}; // the inverted matrix.
    var det =  mat.xAx * mat.yAy - mat.xAy * mat.yAx; // gets the scaling factor called determinate
    rMat.xAx  = mat.yAy / det;
    rMat.xAy  = -mat.xAy / det;
    rMat.yAx  = -mat.yAx / det;
    rMat.yAy  = mat.xAx / det;
    // and invert the origin by moving it along the 90deg rotated axis inversely scaled
    rMat.ox = (mat.yAx * mat.oy - mat.yAy * mat.ox) / det;
    rMat.oy = -(mat.xAx * mat.oy - mat.xAy * mat.ox) / det;     
    return rMat;
}

Putting it all together

Now you can get the information you need.

You have a box

var box = { x : -50, y : -50, w : 100, h : 100 };

and you have a position scale and rotation for that box

var boxPos = {x : 100, y : 100, scaleX : 2, scaleY : 2, rotate : 1};

To render it you need to create a transform, set the context to that matrix and render.

var matrix = recomposeMatrix(boxPos.x, boxPos.y, boxPos.scaleX, boxPos.scaleY, boxPos.rotate);
ctx.setTransform(matrix.xAx, matrix.xAy, matrix.yAx, matrix.yAy, matrix.ox, matrix.oy);
ctx.strokeRect(box.x, box.y, box.w, box.h);

To find out if the mouse (in screen space) is inside the box you want the mouse in local (box coordinates). To do that you need the inverted box matrix, which you apply to the mouse coordinates.

var invMatrix = invertMatrix(matrix);
var mouseLocal = transformPoint(invMatrix, mouse.x, mouse.y);

if(mouseLocal.x > box.x && mouseLocal.x < box.x + box.W && mouseLocal.y > box.y && mouseLocal.y < box.y + box.h){
    // mouse is inside
}

As simple as that. The mouseLocal coordinates are in the boxes space, and thus it is simple geometry to get the relative position to corners and the like.

You may think that is a lot of work to get the mouse relative coordinates. Yes maybe for a single rotated box it is. You could just use absolute screen coordinates. But what if the rotate the world space, and then the box is attached to another object and another that are both scaled rotated and positioned. The transformations of the world, obj1, obj2 and finally your box can be multiplied together to get a transformation matrix for the box. Invert that one matrix and you have the relative position of a screen coordinate.

More transforms

You will need some extra functionality for the transformation matrix so it would be a good idea to write your own matrix class, or you can get one (or bad one as many are very poorly written) from github, or you can use the built in matrix support that most browsers have in the experimental stage and will need prefixes or flags set to use. Find them in MDN