I have a shader that is supposed to work with either alpha blending or alpha testing, but the color values being passed in are premultiplied alpha values.
Is there an easy/standard way to have it produce the "correct" results for both alpha blending and alpha testing when using premultiplied alpha?
For example, if the final result of my shader is RGBA(1,0,0,0.75)
straight alpha and therefore RGBA(0.75,0,0,0.75)
premultiplied alpha, the result should be remain RGBA(1,0,0,0.75)
when alpha testing.
Perhaps one option is "dividing out" the alpha channel if we are alpha testing, i.e. divide the R
by A
above so that we get 0.75/0.75 = 1.0
, our original red value. But this becomes non-deterministic when the alpha goes to 0.
Maybe I can clarify with an example:
Say the back buffer is RGBA(1,1,1,1)
, and i am rendering with a partially transparent green color RGBA(0,1,0,0.75)
(straight alpha). The color is passed in as premultiplied alpha as RGBA(0,0.75,0,0.75)
. When alpha blending, i blend using one + invsrcalpha and get a final color of RGBA(0.25,1,0.25,1)
. So my green channel stays at 100%, because the back buffer had green at 100%, and so did my original green color, so no way to get less than 100% on the green channel.
When Alpha testing, there is no blending going on. Instead, my green color will be rendered straight to the back buffer as RGBA(0,0.75,0,1)
. But the color I wanted was RGBA(0,1,0,1)
.
RGBA(0,1,0,1)
, but if its antialised around the edges of the cutout, some pixels might beRGBA(0,0.75,0,0.75)
in premultiplied alpha. However, since i'm alpha testing and not alpha blending, I want the green color to stay 100% green and not get dark around the edges. Maybe there's a fundamental flaw in the way i'm imagining that alpha cutouts and anti-aliasing should work? \$\endgroup\$