OpenGL blending with source and destination alpha

Question

I'm trying to render a texture with alpha to a frame buffer with alpha. I need the texture's transparency to be controlled by a combination of its own alpha, and the alpha already rendered to the frame buffer.

ie:

Normal transparency:

(src_colour * src_alpha) + (dst_colour * 1-src_alpha)

Required transparency:

(src_colour * src_alpha * dst_alphe) + (dst_colour * 1-(src_alpha * dst_alpha))

Can anyone figure out how to do this?

I've been working on it for 9 hours now :-)

I'm using OpenGL 1.0 with GL11ExtensionPack on Android, so I have access to glBlendFunc and glBlendFuncSeparate. I would prefer not to go the GLES20 route with shaders as that would require a lot of code to be rewritten.

Thanks, Andrew

Answer 1

This is the solution I've used:

1. enable colour and alpha writes
2. glBlendFunc(GL10.GL_SRC_ALPHA, GL10.GL_ONE_MINUS_SRC_ALPHA)
3. render foreground objects
4. disabe alpha writes
5. render shadows
6. render objects that are casting shadows
7. glBlendFunc(GL10.GL_ONE_MINUS_DST_ALPHA, GL10.GL_DST_ALPHA)
8. render background objects

This produces the desired effect of shadows appearing only on foreground objects. Effectively the alpha buffer becomes a stencil buffer with variable opacity.

It's fortunate that my background objects don't have transparency, as if they did I don't think there would be any way to achieve this effect without using shaders.