0
\$\begingroup\$

We are into a game where to build our tile map, we carve on an image with a mask (At first the whole map is an image, and we create holes on it with tiles to create the map and le the background show. It is like substracting instead of adding new tiles in any common tiled based game).

We have done this with just a tile that acts as a mask using stencil buffer. The problem comes when we want to mask with tiles where all their pixels are not opaque, for example a slope. The problem is that it applies the mask as if all the pixels in tile where opaque and we want to apply the mask in two differnet ways:

1) Keep the pixels of the big image where the mask pixels are opaque

2) Mask the pixels of the big image where the mask pixels are transparent (alpha==0)

I have read a lot of info about using stencil and alpha blending but can't make it work. Checking the pipeline order it seems that stencil is applied after the fragment color has been calculated. Is it possible to access the alpha value at stencil stage and write to the stencil as needed to just mask part of the image (tile in my case).

Update to add a visual example: enter image description here

\$\endgroup\$
  • \$\begingroup\$ Can you include an image or animation of the final effect you want to achieve? There may be easier routes to this than selective stencil writing. \$\endgroup\$ – DMGregory Nov 22 '16 at 16:32
  • \$\begingroup\$ @DMGregory I have added an image explaining the process we are following. We are choosing this workflow because it gives artists a lot of freedom when creating platforms (they can paint a full image or several without the need of using tiles). Then just carve on it what they need. \$\endgroup\$ – Notbad Nov 22 '16 at 19:47
1
\$\begingroup\$

Well, I finally fixed this using a discard for the pixels I don't want to be tested by stencil. It is working but discard operations are not the most performant :). Depending on the user case it could be a solution. It is in mine.

Cheers.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.