Capping alpha for sprite blending

Question

In modern OpenGL, I am drawing a series of adjacent sprites using a textured quad for each sprite (as in a brush stroke). What technique could I use to draw the sprites without the alpha value accumulating?

To be clear, let me show an example.

Assume my sprite is a semi-transparent circle, as such:

If I draw two adjacent sprites, I would get this:

Which is very normal, though what I want to get is this:

Where the alpha value of the result is the maximum of the alphas of the source and dest, as if they are part of the same visual layer.

I assume this involves my blend function . Currently it is:

 glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

Which blend function or other technique could I use to achieve the desired result?

Answer 1

(Ab)use the stencil buffer:

1. Clear stencil buffer to 0xff
2. Foreach circle, only write (& blend) into the color & stencil buffers where the stencil buffer is 0xff. If the stencil & depth tests succeed set the stencil value for the fragment to zero.

This way you can only write (& blend) a color pixel once until you clear the stencil buffer again.

Example:

#include <GL/glut.h>

void quad()
{
    glBegin( GL_QUADS );
    glVertex2i( -1, -1 );
    glVertex2i(  1, -1 );
    glVertex2i(  1,  1 );
    glVertex2i( -1,  1 );
    glEnd();
}

void display()
{
    glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );

    glMatrixMode( GL_PROJECTION );
    glLoadIdentity();
    glOrtho( -2, 2, -2, 2, -1, 1 );

    glMatrixMode( GL_MODELVIEW );
    glLoadIdentity();

    // clear stencil buffer to all 1s
    glClearStencil( 0xff );
    glClear( GL_STENCIL_BUFFER_BIT );

    // turn on stencil testing & make sure we don't mask out any bits
    glEnable( GL_STENCIL_TEST );
    glStencilMask( 0xff );

    // only allow fragments through to the color/depth buffers if
    // the stencil buffer at that point is 0xff
    glStencilFunc( GL_EQUAL, 0xff, 0xff );

    // if the stencil/depth tests succeed for a fragment, 
    // zero out the stencil buffer at that point;
    // that way you can only write to a color buffer pixel once
    glStencilOp( GL_KEEP, GL_KEEP, GL_ZERO );

    glEnable( GL_BLEND );
    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

    glPushMatrix();
    glTranslatef( -0.5, -0.5, 0 );
    glColor4ub( 255, 0, 0, 128 );
    quad();
    glPopMatrix();

    glPushMatrix();
    glTranslatef( 0.5, 0.5, 0 );
    glColor4ub( 255, 0, 0, 128 );
    quad();
    glPopMatrix();

    glutSwapBuffers();
}

int main( int argc, char **argv )
{
    glutInit( &argc, argv );
    glutInitDisplayMode( GLUT_RGBA | GLUT_DOUBLE | GLUT_STENCIL );
    glutInitWindowSize( 600, 600 );
    glutCreateWindow( "GLUT" );
    glutDisplayFunc( display );
    glutMainLoop();
    return 0;
}

Answer 2

This is what I ended up doing. It's hard to call it the answer, as I have changed the parameters of the question a bit, but I thought I would mention it for posterity.

Instead of using transparent sprites, I use fully opaque sprites.

I then render these to a texture, and then draw that texture with the desired opacity.