How can I round out the corners of a quad in OpenGL?

Question

Even though I think the image is pretty self-explanatory, I'm going to explain what I am trying to achieve:

I was working on a game, and I needed a GUI framework, that I decided to build myself. As a first thing, I tried rendering a simple 2d quad with an orthographic projection. I was not satisfied with the boxy shapes windows had, so I tried fixing the problem by adding a texture; all this worked until I tried implementing resizing: whenever I changed the window size, the texture was warping, and it did not look good. Now I'm trying to use more than just two triangles, to smooth the corners of the windows. Now, my question is: how can I determine the number of triangles each window should have to have in order to obtain smoother corners? I would like to have some sort of "smooth" variable, that specifies how smooth(how many triangles are used) the window is. Thanks everyone for the help

EDIT:

I was thinking about this: do you guys think that generating the vertices of the window based on a condition in real time would be a good idea?

something like(proto code):

if(smooth == 1)
{
    //define quad vertices
    float vertices[] = 
    {
       bla,bla,
       bla,bla,
       bla,bla....
    }
}

if(smooth == 2)
{
    float vertices[] = 
    {
       bla,bla,
       bla,bla,
       bla,bla....
    }
}

In a way, this would work, but I'm pretty sure there is a much better way to do this...

Answer 1

This is not technically an answer to your question, but is a better work around in my opinion.

You can pass the dimensions of the boxes and the radius of the corners to the fragment shaders and round the corners that way.

Basically, you take the current texture coordinates, multiply each coordinate of it by the dimensions of the window to get the coordinates of the current fragment relative to the window. Then if the distance between this position and each of the edges is less, than the radius of the corner, then you throw it away.

This way you don't need to upload separate vertices for each window.

Sample code in GLSL (I don't know what you're using)

in vec2 a_uv;

uniform vec2 u_dimensions;
uniform float u_radius;

void main(void) {
    vec2 coords = a_uv * u_dimensions;
    if (length(coords - vec2(0) < u_radius ||
        length(coords - vec2(0, u_dimensions.y) < u_radius ||
        length(coords - vec2(u_dimensions.x, 0) < u_radius ||
        length(coords - u_dimensions) < u_radius) {
        discard;
    }
    // Do everything else otherwise
}

Answer 2

The main problem with drawing round 2D objects using 3D geometry is that there's no real fixed mapping between the pixels on the screen and the 3D world-space. Making sure they look at all resolutions and various transformations can get really tricky real fast, and you don't really want to spend time on something that doesn't really improve the game quality. But you're right that you can't stretch a single texture and get a good effect either.

One typical way this is done in 2D GUIs is 9-sliced scaling. Instead of having one logical texture, you have nine - one for each corner, one for each edge and one for the rest. The corners aren't scaled at all, only moved to their proper place. The edges are repeated (or sometimes scaled) in their proper direction (e.g. the top edge will be repeated in the X-direction). The center is repeated in both directions. This allows you to resize the window at will.

If you want to keep 3D geometry, pixel shaders sound like the best way to go - after all, you're trying to scale the effect to the screen, not the world-space. Drawing a simple circle section is trivial with just a bit of math, though making sure you get good looking anti-aliasing etc. can make this trickier. The main advantage over the texture method is you can arbitrarily scale it without stretching the textures; the main disadvantage is that you'll only ever do very simple patterns (and the more complicated the pattern, the more taxing to the GPU). Of course, you can also combine both approaches - have a texture that's applied to the "solid" parts of the window but not the transparent parts, and repeated as needed.

Answer 3

I'll extend on Bálint's excellent answer by adding my own idea of a possible fragment-shaded approach:

in vec2 vUV;

uniform vec2 uExtents;
uniform float uRadius;

float calcDistance() {
    vec2 coords = abs(vUV) * (uExtents + uRadius);
    vec2 delta = max(coords - uExtents, 0);
    return length(delta);
}

void main() {
    float dist = calcDistance();
    if (dist > uRadius) discard;
    /* ... */
}

Here I take as inputs the vertex UV vUV ∈ [-1, 1] × [-1, 1], rectangle extents uExtents = (width/2, height/2), and the corner radius uRadius. From these we produce a distance to the rectangle where 0 represents inside and all other (positive) values represent outside.