What is the simplest way to implement a shader to draw a circular portion of a full screen texture where the circle is dynamically resized as part of an animation using LibGDX?

Is a shader even the best place to do this kind of thing?

I'm sure you could do this in software using a Pixmap but that would be slow. In the end I managed to get something working using the OpenGL depth buffer, a ShapeRenderer to draw a circular mask and a SpriteBatch to draw the texture but it feels messy and awkward.


1 Answer 1


Alpha mask texture approach

In case your circle would always be the same size, using a second alpha mask texture would be the way to go. You would make it a grayscale mask texture and use its value as the alpha value while drawing. Using a texture had the benefit of having anti-aliasing built into the mask (ie. at the borders of your circle, the pixels can be partially opaque, rather than either fully opaque or transparant).

Fragment discarding approach

But as you mention it should be animated, this is not a possibility. A shader approach using fragment discarding would be to pass the circle's parameters (center position [x,y] and radius [r]) as uniforms to the shader. As you want to render a specific circular region of your texture, you would express these in terms of UVs. Then, in the frament shader, you check whether the fragment's UVs fall within the circle's boundaries (aka. if their distance to the center of the circle is larger than the radius of the circle). If so, you can discard the fragment (this means nothing will be rendered). Note that this is a binary approach; pixels get drawn or they don't get drawn, they cannot be partially opaque. This can cause jaggy edges, aka aliasing.

Alpha calculation approach

To get rid of aliasing, you could adapt the previous approach. Rather than using the circle's parameters to check whether to discard the fragment, you could also use it to calculate an alpha value. To get rid of aliasing, you could specify to radii rather than one [r_inner, r_outer]. Everything inside the inner radius will be fully opaque and everything outside the outer radius will be fully transparant. For fragments inbetween the radii, you could apply linear interpolation. This will give you a smoother border. For example, using a 2 pixel difference between the inner and outer radius will give you a 2 pixel border around your circle where pixels smoothly transition from fully opaque to fully transparant. In short, you would calculate the alpha as (pseudocode, not actual GLSL):

distance = sqrt(pwr(u - circle_x, 2) +  pwr(u - circle_y, 2))
alpha = ((distance - r_inner) / (r_outer - r_inner));
clamp(0, 1, alpha)

Generate a circle using polygons

You can also use polygons to define the circle. Rather than drawing your texture on a quad, draw an actual circle. Calculating the vertices of your polygons to form a circle is very simple, as you just need to use points at regular angular intervals around the center point, at the desired radius. In the same way calculate the UVs by calculating the respective points on the texture.

Dynamically calculated mask

Another approach, resembling the alpha mask texture approach would be to create your masking texture on-the-fly. Reserve a texture for the mask, render it fully black. Draw a white circle on it. Then use this texture as an alpha mask.

In short, a clean approach that works for all shapes is to use an alpha mask. This is not suitable for dynamic animation however. The cleanest approach that allows for animation is to generate a circle of polygons to draw on. An good alternative is to calculate the alpha in the fragment shader.

  • 1
    \$\begingroup\$ Wow. That was a better answer than I'd hoped for. I've tried the fragment discarding method but was using gl_FragCoord instead of texture coordinates as you suggested, so my circle was not being scaled correctly. I might take a look at texturing a circular polygon next but it looks like I'll have to get head around UV mapping. Thanks. \$\endgroup\$
    – Matt
    Commented Apr 17, 2015 at 17:49
  • \$\begingroup\$ Discarding also kills early-z :/ \$\endgroup\$
    – Jon
    Commented Apr 18, 2015 at 4:20
  • \$\begingroup\$ Is that good or bad? \$\endgroup\$
    – Matt
    Commented Apr 18, 2015 at 17:51
  • \$\begingroup\$ I think he means it gets rid of z-buffering problems (a good thing). The z-buffer stores the depth of drawn fragments - their distance to the camera. Before a new fragment is rendered, the z-buffer is consulted to check whether something else was drawn in front of it (closer to the camera). If so, the fragment is invisible, so it will not be drawn, speeding up the drawing process and making sure only front-most objects are visible. This causes issues when drawing semi-transparant pixels. Using the fragment discarding approach, you won't run into this problem, as the z-buffer is not written to. \$\endgroup\$ Commented Apr 19, 2015 at 11:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .