How to cut a “hole” inside a rectangular Sprite to see the Sprite underneath? (ActionScript 3)

Question

Everytime I google this question I see confusing information about masks and blends, none of which seems to directly apply to what I think should be an easy thing...

There are three Sprites involved here...the lowest layer sprite is pretty much a background. I want to overlay a translucent Sprite on top of the background and then I want the third, top-most Sprite to act as a hole, so that the area inside the third Sprite is completely transparent, so that the background sprite is completely visible.

How would I go about doing this dynamically (ie dynamically drawing the masking sprite and hole using the Actionscript graphics calls)?

Answer 1

I know it's been a long time ago, but just for people who might look for the same problem. It's actually quite easy (as long as draw your Sprite with the graphics class).

1) Inflexible hole cutting:

this.graphics.beginFill(0x666666);
this.graphics.drawRect(0,0,256, 256);
this.graphics.drawCircle(128,128,32);
this.graphics.endFill();

this will create a rectangle of 256 by 256 with a 64px hole in it.

2) Flexible hole cutting:

Obviously this will not work when you're not using the graphics class. In That case I would go with BlendMode.ERASE .

var spr:Sprite = new Sprite();
var msk:Sprite = new Sprite();

addChild(spr);
spr.addChild(msk)

spr.graphics.beginFill(0x666666);
spr.graphics.drawRect(0,0,256, 256);
spr.graphics.endFill();

msk.graphics.beginFill(0x000000);
msk.graphics.drawEllipse(0,0,64,64);
msk.graphics.endFill();
msk.x = 128;
msk.y = 128;

spr.blendMode = BlendMode.LAYER;
msk.blendMode = BlendMode.ERASE;

When using BlendMode.ERASE you must ALWAYS set the parentcontainer's blendmode to BlendMode.LAYER, otherwise it won't work.

I hope this might help someone

Answer 2

If I recall it correctly, if you draw two overlapping objects on the same graphics without stopping the fill, they will work pretty much like blendmode.erase. Pseudo-code would look like this:

graphics.beginFill (any color, any alpha);
graphics.drawRect(0,0,256, 256);
graphics.drawRect(128,128,64, 64);
graphics.endfill ();

If this actually works (and I think I used this solution many years ago and just recently saw that SVG files work this way), this should give you a much better performance.

EDIT: I have just used this myself in one of my games.

Answer 3

Not an AS developer, but take a look at this: http://code.google.com/p/gpcas/

This should enable you to do boolean "cutout" operations on two polygons.

Answer 4

OK, what I want can be done as simple as calling drawRect twice on the translucent layer...once to draw the translucent rectangle, and again to draw the "hole"

Details here: http://www.actionscript.org/forums/showthread.php3?p=965632

Not exactly a mask, but good enough for my purposes.

Answer 5

Check out this method, it allows for an optional "hole" parameter. This is found in the ProgrammaticSkin class of the flex framework.

http://opensource.adobe.com/svn/opensource/flex/sdk/trunk/frameworks/projects/framework/src/mx/skins/ProgrammaticSkin.as

/**
     *  Programatically draws a rectangle into this skin's Graphics object.
     *
     *  <p>The rectangle can have rounded corners.
     *  Its edges are stroked with the current line style
     *  of the Graphics object.
     *  It can have a solid color fill, a gradient fill, or no fill.
     *  A solid fill can have an alpha transparency.
     *  A gradient fill can be linear or radial. You can specify
     *  up to 15 colors and alpha values at specified points along
     *  the gradient, and you can specify a rotation angle
     *  or transformation matrix for the gradient.
     *  Finally, the rectangle can have a rounded rectangular hole
     *  carved out of it.</p>
     *
     *  <p>This versatile rectangle-drawing routine is used by many skins.
     *  It calls the <code>drawRect()</code> or
     *  <code>drawRoundRect()</code>
     *  methods (in the flash.display.Graphics class) to draw into this
     *  skin's Graphics object.</p>
     *
     *  @param x Horizontal position of upper-left corner
     *  of rectangle within this skin.
     *
     *  @param y Vertical position of upper-left corner
     *  of rectangle within this skin.
     *
     *  @param width Width of rectangle, in pixels.
     *
     *  @param height Height of rectangle, in pixels.
     *
     *  @param cornerRadius Corner radius/radii of rectangle.
     *  Can be <code>null</code>, a Number, or an Object.
     *  If it is <code>null</code>, it specifies that the corners should be square
     *  rather than rounded.
     *  If it is a Number, it specifies the same radius, in pixels,
     *  for all four corners.
     *  If it is an Object, it should have properties named
     *  <code>tl</code>, <code>tr</code>, <code>bl</code>, and
     *  <code>br</code>, whose values are Numbers specifying
     *  the radius, in pixels, for the top left, top right,
     *  bottom left, and bottom right corners.
     *  For example, you can pass a plain Object such as
     *  <code>{ tl: 5, tr: 5, bl: 0, br: 0 }</code>.
     *  The default value is null (square corners).
     *
     *  @param color The RGB color(s) for the fill.
     *  Can be <code>null</code>, a uint, or an Array.
     *  If it is <code>null</code>, the rectangle not filled.
     *  If it is a uint, it specifies an RGB fill color.
     *  For example, pass <code>0xFF0000</code> to fill with red.
     *  If it is an Array, it should contain uints
     *  specifying the gradient colors.
     *  For example, pass <code>[ 0xFF0000, 0xFFFF00, 0x0000FF ]</code>
     *  to fill with a red-to-yellow-to-blue gradient.
     *  You can specify up to 15 colors in the gradient.
     *  The default value is null (no fill).
     *
     *  @param alpha Alpha value(s) for the fill.
     *  Can be null, a Number, or an Array.
     *  This argument is ignored if <code>color</code> is null.
     *  If <code>color</code> is a uint specifying an RGB fill color,
     *  then <code>alpha</code> should be a Number specifying
     *  the transparency of the fill, where 0.0 is completely transparent
     *  and 1.0 is completely opaque.
     *  You can also pass null instead of 1.0 in this case
     *  to specify complete opaqueness.
     *  If <code>color</code> is an Array specifying gradient colors,
     *  then <code>alpha</code> should be an Array of Numbers, of the
     *  same length, that specifies the corresponding alpha values
     *  for the gradient.
     *  In this case, the default value is <code>null</code> (completely opaque).
     *
     *  @param gradientMatrix Matrix object used for the gradient fill. 
     *  The utility methods <code>horizontalGradientMatrix()</code>, 
     *  <code>verticalGradientMatrix()</code>, and
     *  <code>rotatedGradientMatrix()</code> can be used to create the value for 
     *  this parameter.
     *
     *  @param gradientType Type of gradient fill. The possible values are
     *  <code>GradientType.LINEAR</code> or <code>GradientType.RADIAL</code>.
     *  (The GradientType class is in the package flash.display.)
     *
     *  @param gradientRatios (optional default [0,255])
     *  Specifies the distribution of colors. The number of entries must match
     *  the number of colors defined in the <code>color</code> parameter.
     *  Each value defines the percentage of the width where the color is 
     *  sampled at 100%. The value 0 represents the left-hand position in 
     *  the gradient box, and 255 represents the right-hand position in the 
     *  gradient box. 
     *
     *  @param hole (optional) A rounded rectangular hole
     *  that should be carved out of the middle
     *  of the otherwise solid rounded rectangle
     *  { x: #, y: #, w: #, h: #, r: # or { br: #, bl: #, tl: #, tr: # } }
     *
     *  @see flash.display.Graphics#beginGradientFill()
     *  
     *  @langversion 3.0
     *  @playerversion Flash 9
     *  @playerversion AIR 1.1
     *  @productversion Flex 3
     */
    protected function drawRoundRect(
                            x:Number, y:Number, width:Number, height:Number,
                            cornerRadius:Object = null,
                            color:Object = null,
                            alpha:Object = null,
                            gradientMatrix:Matrix = null,
                            gradientType:String = "linear",
                            gradientRatios:Array /* of Number */ = null,
                            hole:Object = null):void
    {
        var g:Graphics = graphics;

        // Quick exit if weight or height is zero.
        // This happens when scaling a component to a very small value,
        // which then gets rounded to 0.
        if (width == 0 || height == 0)
            return;

        // If color is an object then allow for complex fills.
        if (color !== null)
        {
            if (color is uint)
            {
                g.beginFill(uint(color), Number(alpha));
            }
            else if (color is Array)
            {
                var alphas:Array = alpha is Array ?
                                   alpha as Array :
                                   [ alpha, alpha ];

                if (!gradientRatios)
                    gradientRatios = [ 0, 0xFF ];

                g.beginGradientFill(gradientType,
                                    color as Array, alphas,
                                    gradientRatios, gradientMatrix);
            }
        }

        var ellipseSize:Number;

        // Stroke the rectangle.
        if (!cornerRadius)
        {
            g.drawRect(x, y, width, height);
        }
        else if (cornerRadius is Number)
        {
            ellipseSize = Number(cornerRadius) * 2;
            g.drawRoundRect(x, y, width, height, 
                            ellipseSize, ellipseSize);
        }
        else
        {
            GraphicsUtil.drawRoundRectComplex(g,
                                   x, y, width, height,
                                   cornerRadius.tl, cornerRadius.tr,
                                   cornerRadius.bl, cornerRadius.br);
        }

        // Carve a rectangular hole out of the middle of the rounded rect.
        if (hole)
        {
            var holeR:Object = hole.r;
            if (holeR is Number)
            {
                ellipseSize = Number(holeR) * 2;
                g.drawRoundRect(hole.x, hole.y, hole.w, hole.h, 
                                ellipseSize, ellipseSize);
            }
            else
            {
                GraphicsUtil.drawRoundRectComplex(g,
                                       hole.x, hole.y, hole.w, hole.h,
                                       holeR.tl, holeR.tr, holeR.bl, holeR.br);
            }   
        }

        if (color !== null)
            g.endFill();
    }       
}

Answer 6

In case someone needs to have a gradient hole, it's possible using trick from Raimundas. Let's say you want elliptical gradient coming from inside to the outside (inner radius alpha = 0, outer radius alpha = 1).

First draw the background without the ellipse

var gfx:Graphics;
gfx.clear();
gfx.beginFill(color, alpha );
gfx.drawRect(0, 0, width, height); // fullscreen

gfx.drawEllipse(e_x, e_y, e_width, e_height); // trick - drawn in the same beginFill

gfx.endFill();

Then draw the gradient ellipse, inside the result hole!

var mat : Matrix = new Matrix();
mat.createGradientBox(width, height, 0, e_x, e_y);

gfx.lineStyle(); // I think it resets lineStyle
gfx.beginGradientFill(GradientType.RADIAL, colors, alphas, ratios, mat);
gfx.drawEllipse(e_x, e_y, width, height);
gfx.endFill();

The ellipse will match hole exactly, pixel-perfect.