# How can I round out the corners of a quad in OpenGL?

Even though I think the image is pretty self-explanatory, I'm going to explain what I am trying to achieve:

I was working on a game, and I needed a GUI framework, that I decided to build myself. As a first thing, I tried rendering a simple 2d quad with an orthographic projection. I was not satisfied with the boxy shapes windows had, so I tried fixing the problem by adding a texture; all this worked until I tried implementing resizing: whenever I changed the window size, the texture was warping, and it did not look good. Now I'm trying to use more than just two triangles, to smooth the corners of the windows. Now, my question is: how can I determine the number of triangles each window should have to have in order to obtain smoother corners? I would like to have some sort of "smooth" variable, that specifies how smooth(how many triangles are used) the window is. Thanks everyone for the help

EDIT:

I was thinking about this: do you guys think that generating the vertices of the window based on a condition in real time would be a good idea?

something like(proto code):

if(smooth == 1)
{
float vertices[] =
{
bla,bla,
bla,bla,
bla,bla....
}
}

if(smooth == 2)
{
float vertices[] =
{
bla,bla,
bla,bla,
bla,bla....
}
}


In a way, this would work, but I'm pretty sure there is a much better way to do this...

• Do you want to do this with geometry, or would 9-slicing be an option?
– user35344
Commented Jan 31, 2018 at 11:45
• @Tyyppi_77 i'd like to do it with the geometry
– user100681
Commented Jan 31, 2018 at 12:06
• @Tyyppi_77 but slicing could be an option :D haha
– user100681
Commented Jan 31, 2018 at 13:11
• Geometry is generally not such a great idea. Tiny triangles cause useless fragment processing and ROP and as such are just as expensive (or more expensive) than having a fragment shader with discard or just using a simple texture as alpha mask in a 9-slice (which is much more flexible and easily skinnable without technical knowledge, or computional overhead). Use a distance field texture if you want to be fancy and feel like corners must be super, super round and perfect regardless of resolution. Commented Jan 31, 2018 at 16:01

This is not technically an answer to your question, but is a better work around in my opinion.

You can pass the dimensions of the boxes and the radius of the corners to the fragment shaders and round the corners that way.

Basically, you take the current texture coordinates, multiply each coordinate of it by the dimensions of the window to get the coordinates of the current fragment relative to the window. Then if the distance between this position and each of the edges is less, than the radius of the corner, then you throw it away.

This way you don't need to upload separate vertices for each window.

Sample code in GLSL (I don't know what you're using)

in vec2 a_uv;

uniform vec2 u_dimensions;

void main(void) {
vec2 coords = a_uv * u_dimensions;
if (length(coords - vec2(0) < u_radius ||
length(coords - vec2(0, u_dimensions.y) < u_radius ||
length(coords - vec2(u_dimensions.x, 0) < u_radius ||
length(coords - u_dimensions) < u_radius) {
}
// Do everything else otherwise
}

• I will definitely try this out, thanks ;) i was trying to imitate dear Imgui, which uses triangles, instead of smoothing corners via fragment shader(that's why I wanted to use vertices :P)
– user100681
Commented Jan 31, 2018 at 12:14
• this works, but what if i don't want to use textures?
– user100681
Commented Jan 31, 2018 at 13:51
• This approach doesn't use textures?
– user35344
Commented Jan 31, 2018 at 14:00
• @GabrieleVierti There are no textures, you don't need a texture to have texture coordinates associated with vertices. In vec2 a_uv; would be a vertex attribute for each vertex corner of your gui box, IE (1,1) (1,0) (0,1) (0,0). GLSL tells the gpu to automatically interpolate between those values for each sub-rectangle position. Commented Jan 31, 2018 at 14:14
• Sure haha i wasn't thinking :D
– user100681
Commented Jan 31, 2018 at 15:17

The main problem with drawing round 2D objects using 3D geometry is that there's no real fixed mapping between the pixels on the screen and the 3D world-space. Making sure they look at all resolutions and various transformations can get really tricky real fast, and you don't really want to spend time on something that doesn't really improve the game quality. But you're right that you can't stretch a single texture and get a good effect either.

One typical way this is done in 2D GUIs is 9-sliced scaling. Instead of having one logical texture, you have nine - one for each corner, one for each edge and one for the rest. The corners aren't scaled at all, only moved to their proper place. The edges are repeated (or sometimes scaled) in their proper direction (e.g. the top edge will be repeated in the X-direction). The center is repeated in both directions. This allows you to resize the window at will.

If you want to keep 3D geometry, pixel shaders sound like the best way to go - after all, you're trying to scale the effect to the screen, not the world-space. Drawing a simple circle section is trivial with just a bit of math, though making sure you get good looking anti-aliasing etc. can make this trickier. The main advantage over the texture method is you can arbitrarily scale it without stretching the textures; the main disadvantage is that you'll only ever do very simple patterns (and the more complicated the pattern, the more taxing to the GPU). Of course, you can also combine both approaches - have a texture that's applied to the "solid" parts of the window but not the transparent parts, and repeated as needed.

• See gamedev.stackexchange.com/a/153787/99481, where none of this is actually a problem and using what you propose is both more computationally expensive and more complicated. Commented Jan 31, 2018 at 14:30
• Thanks for the asnwer! I might try this out, i just have one question though... how would i go for applying textures to the edges and corners of the window? Shouldn't i use more textured quads? Or have more texture coordinates relative to the window?
– user100681
Commented Jan 31, 2018 at 15:24
• @GabrieleVierti Multiple textured quads work great, yeah. You can also simulate it in the pixel shader, but that's probably more trouble than it's worth. Commented Jan 31, 2018 at 17:11
• @snb I have no idea what your point is - he's also using a pixel shader, not geometry. Are you saying I should copy other answers' solution? Are you trying to say that drawing simple 2D quads is an overkill for rendering a 2D UI, while using 3D geometry is the perfect solution? Or that I should only present one solution? The 9-slice approach is a lot more flexible - it allows arbitrary window graphics. Maybe Gabriele needs that flexibility, maybe he doesn't. The answers are there for other people too anyway, though. Commented Jan 31, 2018 at 17:28
• @snb Mind you, I wouldn't have included the PS example at all if I realised that's what's Bálint describing - a bit of a terminology mixup, I've never worked much with "modern" OpenGL. But given that it's right there in my answer, your comment is all the weirder - you're pointing to the exact same solution and complaining I dismissed it... Commented Jan 31, 2018 at 17:32

in vec2 vUV;

uniform vec2 uExtents;

float calcDistance() {
vec2 coords = abs(vUV) * (uExtents + uRadius);
vec2 delta = max(coords - uExtents, 0);
return length(delta);
}

void main() {
float dist = calcDistance();

Here I take as inputs the vertex UV vUV ∈ [-1, 1] × [-1, 1], rectangle extents uExtents = (width/2, height/2), and the corner radius uRadius. From these we produce a distance to the rectangle where 0 represents inside and all other (positive) values represent outside.