# How to fix texture edge artefacts?

This issue has been annoying me for a long time now and even after reading a lot of articles about it, I am still unable to fix the issue.

First, to my setup. I'm using LWJGL for a 2D project, rendering in immediate mode (yes, I know, FBOs, but I feel like it's too late to change it now). Textures can be scaled, rotated tinted and cropped in any way and are read from packed spritesheets that are multiples of 2 (the one in question is 512x1024 for example).

The Problem

Because there is no smoothing applied to the textures, scaled and rotated textures look quite ugly, showing jagged lines and black outlines. I tried improving it with using glEnable(GL_POLYGON_SMOOTH) (even though I know now that this is apparantly a bad practice, but I have no idea how to add multisampling to immediate mode rendering, other sites even state that multisampling is probably overkill for 2D games, which leaves me even more confused). This is what it looks like, with and without POLYGON_SMOOTH:

Quite clearly, there are multiple issues visible:

1. With POLYGON_SMOOTH disabled, there are horribly jagged lines and dark edges (that aren't there in the texture)
2. With POLYGON_SMOOTH enabled, the edges are no longer jagged, but the dark edges are still there and now there are those diagonal lines. After reading up on that a bit, I found out that it has something to do with how graphics cards render quads, which is as two triangles, and when smoothing is applied, this happens. To test that out, instead of drawing a single quad I draw two triangles which indeed moves the diagonal lines around.

The dark edges seem to be caused by something like like pre-multiplied alpha (according to this), but when I check the raw pixel data, it doesn't look pre-multiplied to me. A completely white texture with 0.5f alpha for example is represented as (in ARGB) 0x80FFFFFF. So applying the "fix" (using glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA) instead of the default glBlendFunc(GL_SRC, GL_ONE_MINUS_SRC_ALPHA)suggested in the article doesn't really work, because my textures aren't premultiplied alpha, so alpha components don't work anymore (same texture, black background):

Texture without alpha:

I do use custom shaders, so I guess I probably could make alpha work in some way by passing it to the shader and then multiplying the color by it, would that fix the dark edges (which would enable me to use GL_ONE, the "better" blend function)? That would still leave me with the issue of the diagonal lines though, which I seem to be able to move around, but not remove.

• Don't use POLYGON_SMOOTH as you just can't get proper antialiasing with that. Instead enable antialiasing for the surface. Check LWJGL documentation on how to enable that. – msell Oct 25 '15 at 12:45
• @msell Thanks for your comment. What exactly is the problem with POLGYON_SMOOTH? Why is it still there if it doesn't work properly? What do you mean with surface anti aliasing? I never heard of that before, and a simply googling for it doesn't seem to give me proper results, just general threads about antialiasing. – flotothemoon Oct 25 '15 at 12:48
• POLYGON_SMOOTH is there for backwards compatibility and those rare cases when someone knows where and how to apply it – Kromster says support Monica Oct 25 '15 at 15:53
• I mean MSAA. You didn't specify the version of LWJGL you are using. In LWJGL 2 you enable it by providing number of samples for PixelFormat when calling Display.create(...),. LWJGL 3 uses GLFW for window management, where you enable multisampling with glfwWindowHint(GLFW_SAMPLES, ...). – msell Oct 26 '15 at 5:57
• @msell Sorry, forgot to add that. I'm using LWJGL 2.9.0. – flotothemoon Oct 26 '15 at 9:33