3
\$\begingroup\$

I am trying to create an image effect where only a part of the image has the effect applied to.

One way I can think of to do this is to define the portion of the image using the stencil buffer.

Stencil Mask Shader:

Stencil 
{
    Ref 1
    Comp always
    Pass replace
}

Image Effect Shader:

Stencil
{
    Ref 1
    Comp equal
    Pass keep
}

OnRenderImage Function

void OnRenderImage(RenderTexture source, RenderTexture destination)
{
    saturationPassMat = CheckShaderAndCreateMaterial(saturationShader, saturationPassMat);

    saturationPassTex = RenderTexture.GetTemporary(source.width, source.height, 24);
    Graphics.SetRenderTarget(saturationPassTex.colorBuffer, source.depthBuffer);

    Desaturate(source, saturationPassTex, saturation);

    Graphics.Blit(saturationPassTex, destination);

}

Applying the stencil portion of the image effect shader to any other object produces a desirable stencil effect (aka: it works).

The Problem:

By the time OnRenderImage is called, it seems that the stencil buffer has already been cleared. Changing Ref to 0 in the Image Effect shader will apply the effect to the entire screen. Any other value will result in a black screen. This can only mean that the stencil buffer has been cleared to 0.

Is there a way to prevent the stencil buffer from clearing? Or is this the right method to use the stencil buffer with image effects? Or perhaps there are other solutions that are still efficient, without the use of Stencils?

\$\endgroup\$
1
\$\begingroup\$

After 3 sleepless nights, here is a somewhat hacky solution in case no one else is able to provide a better answer.

Turns out, blitting directly to destination works in forward rendering.

void OnRenderImage(RenderTexture source, RenderTexture destination)
{
    Graphics.Blit(source, destination, stencilMat);    
}

Many down sides though. Firstly, any image effect applied prior to this script will be culled by the stencil buffer for some reason. Any image effect applied after will stop this from working. Try it and you'll know what I mean.

To get around this, one can use a second camera, setting the render target to a custom render texture. The code above stays the same, and should be attached to the second camera instead.

On the main camera, you can pass this render texture into your shader and use it as a mask. Setting the rendertarget does not seem to work for me at all, no matter what I try.

stencilMat.SetTexture("_MaskTex", cam2.GetMaskTex());

Since in my case I'm doing a desaturation, the shader would look something like:

col.rgb = lerp((col.r + col.g + col.b), col.rgb, maskCol.rgb);
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.