Seamless tilemap rendering (borderless adjacent images)

Question

I have a 2D game engine that draws tilemaps by drawing tiles from a tileset image. Because by default OpenGL can only wrap the entire texture (GL_REPEAT), and not just part of it, each tile is split off in to a separate texture. Then regions of the same tile are rendered adjacent to each other. Here's what it looks like when it's working as intended:

However as soon as you introduce fractional scaling, seams appear:

Why does this happen? I thought it was due to linear filtering blending the borders of the quads, but it still happens with point filtering. The only solution I've found so far is to ensure all positioning and scaling only happens at integer values, and use point filtering. This can degrade the visual quality of the game (particularly that sub-pixel positioning no longer works so motion is not so smooth).

Things I have tried/considered:

• antialiasing reduces, but does not entirely eliminate, the seams
• turning off mipmapping, has no effect
• render each tile individually and extrude the edges by 1px - but this is a de-optimisation, since it can no longer render regions of tiles in one go, and creates other artefacts along the edges of areas of transparency
• add a 1px border around source images and repeat the last pixels - but then they are no longer power-of-two, causing compatibility problems with systems without NPOT support
• writing a custom shader to handle tiled images - but then what would you do differently? GL_REPEAT should be grabbing the pixel from the opposite side of the image at the borders, and not pick transparency.
• the geometry is exactly adjacent, there are no floating point rounding errors.
• if the fragment shader is hard coded to return the same color, the seams disappear.
• if the textures are set to GL_CLAMP instead of GL_REPEAT, the seams disappear (although the rendering is wrong).
• if the textures are set to GL_MIRRORED_REPEAT, the seams disappear (although the rendering is wrong again).
• if I make the background red, the seams are still white. This suggests it's sampling opaque white from somewhere rather than transparency.

So the seams appear only when GL_REPEAT is set. For some reason in this mode only, at the edges of the geometry there is some bleed/leakage/transparency. How can that be? The entire texture is opaque.

Answer 1

The seams are correct behaviour for sampling with GL_REPEAT. Consider the following tile:

Sampling on the edges of the tile using fractional values mixes colors from the edge and the opposite edge, resulting in wrong colors: The left edge should be green, but is a mix from green and beige. The right edge should be beige, but is a mixed color as well. Especially the beige lines on the green background are very visible, but if you look closely you can see the green bleeding into the edge:

antialiasing reduces, but does not entirely eliminate, the seams

MSAA works by taking more samples around polygon edges. Samples taken left- or right of the edge will be "green" (again, considering the left edge of the first picture), and only one sample will be "on" the edge, partialy sampling from the "beige" area. When the samples are mixed the effect will be reduced.

Possible solutions:

In any case you should switch to GL_CLAMP to prevent bleeding from the pixel at the opposite side of the texture. Then you have three options:

1. Enable anti-aliasing to smooth the transition between the tiles a bit.

2. Set the texture filtering to GL_NEAREST. This gives all pixels hard edges, so polygon/sprite edges become indistinguishable, but it is obviously changes the style of the game quite a bit.

3. Add the already discussed 1px border, just make sure the border has the color of the adjacent tile (and not the color of the opposite edge).

   • This might also be a good time to switch to a texture atlas (just make it bigger if you are concerned about NPOT support).
   • This is the only "perfect" solution, enabling GL_LINEAR-like filtering across the edges of polygons/sprites.

Answer 2

Because by default OpenGL can only wrap the entire texture (GL_REPEAT), and not just part of it, each tile is split off in to a separate texture. Then regions of the same tile are rendered adjacent to each other.

Consider the display of an single, ordinary textured quad in OpenGL. Are there any seams, at any scale? No, never. Your goal is to get all your scene tiles packed tightly onto a single texture, then send that to the screen. EDIT To clarify this further: If you have discrete bounding vertices on each tile quad, you will have seemingly-unjustified seams under most circumstances. It's how GPU hardware works... floating point errors create these gaps based on the current perspective... errors which are avoided on a unified manifold, since if two faces are adjacent within the same manifold (submesh), the hardware will render them without seams, guaranteed. You've seen this in countless games and apps. You must have one tightly-packed texture on a single submesh without doubled vertices to avoid this once and for all. It's an issue that comes up time and again on this site and elsewhere: if you don't merge the corner vertices of your tiles (every 4 tiles share a corner vertex), expect seams.

(Consider that you may not even need vertices except at the four corners of the entire map... depends on your approach to shading.)

To solve: render (at 1:1) all of your tile textures into an FBO/RBO with no gaps, then send that FBO to the default framebuffer (the screen). Because the FBO itself is basically a single texture, you cannot end up with gaps on scaling. All texel boundaries that don't fall on a screen pixel boundary are going to be blended if you're using GL_LINEAR.... which is exactly what you want. This is the standard approach.

This also opens up a number of different routes to scaling:

• scaling the size of the quad to which you will render the FBO
• changing the UVs on that quad
• fiddling with camera settings

Answer 3

If the images should appear as one surface, render these as one surface texture (scale 1x) on one 3D mesh/plane. If you render each as a separate 3D object it will always have seams due to rounding errors.

The answer by @Nick-Wiggill is correct, i think you misunderstood it.

Answer 4

2 possibilities that might be worth trying:

1. Use GL_NEAREST instead of GL_LINEAR for your texture filtering in. (As pointed out by Andon M. Coleman)

   GL_LINEAR will (in effect) blur the image ever so slightly (interpolating between the nearest pixels), which allows for a smooth transition in color from one pixel to the next. This can help make textures look better in many cases, since it prevents your textures from having those blocky pixels all over it. It also makes it so that a little bit of brown from one pixel of one tile over can get blurred over to the adjacent green pixel of the adjacent tile.

   GL_NEAREST just finds the nearest pixel and uses that color. No interpolation. As a result, it's actually a little faster to.

2. Shrink the texture coordinates for each tile a little.

   Sort of like adding that extra pixel around each tile as a buffer, except instead of expanding the tile on the image, you shrink the tile in the software. 14x14px tiles while rendering instead of 16x16px tiles.

   You might even be able to get away with 14.5x14.5 tiles without too much brown being mixed in.

--edit--

As shown in the image above, you can still easily use a power of two texture. So you can support hardware that doesn't support NPOT textures. What changes is your texture coordinates, instead of going from (0.0f, 0.0f) to (1.0f, 1.0f) You'd go from (0.03125f, 0.03125f) to (0.96875f, 0.96875f).

That puts your tex-coords slightly inside the tile which reduces the effective resolution in game (though not on hardware so that you still have power-of-two textures), however should have the same rendering effect as expanding the texture.

Answer 5

Here is what I think that might be happening: Where two tiles collide, the x component of their edges should be equal. If that is not true, they might get rounded into the opposite direction. So make sure that they have exactly the same x value. How do you ensure that this happens? You might try to, lets say, multiply by 10, round and then divide by 10 (only do this on the CPU, before your vertices are going into the vertex shader). That should give correct results. Also, do not draw tile by tile using transformation matrices, but put them in a batch VBO to make sure that the wrong results are not coming from the lossy IEEE floating point system in combination with matrix multiplications.

Why do I suggest this? Because from my experience, if the vertices have the exact same coordinates when they come out of the vertex shader, filling the corresponding triangles will create seamless results. Keep in mind that something that is mathematically correct, might be a bit off due to IEEE. The more calculations you perform on your numbers, the less accurate the result will be. And yes, matrix multiplications require quite some operations, that might be done in only a multiplication and an addition when creating your VBO, which will give more accurate results.

What also might be the problem is that you are using a spritesheet (also called atlas) and that when sampling the texture, the pixels of the adjacent tile texture gets picked. To ensure that this doesn't happen is to create a tiny border in the UV mappings. So, if you have a 64x64 tile, your UV mapping should cover a bit less. How much? I think I used in my games 1 quarter of a pixel at each side of the rectangle. So offset your UV's by 1/(4*widthOfTheAtlas) for x components and 1/(4*heightOfTheAtlas) for y components.

Answer 6

To solve this in 3D space, I mixed texture arrays with Wolfgang Skyler's suggestion. I put a 8-pixel border around my real texture for each tile (120x120, 128x128 total) which, for each side, which I could fill with "wrapped" image or the side just being extended. The sampler reads this area when it is interpolating the image.

Now with filtering and mipmapping, the sampler can still easily read past the entire 8 pixel border. To catch that small problem (I say small because it only happens on a few pixels when geometry is really skewed or far away), I split the tiles into a texture array, so each tile has its own texture space and can use clamping on the edges.

In your case (2D/flat), I would certainly go with rendering a pixel perfect scene, and then scaling the desired portion of the result into the viewport, as suggested by Nick Wiggil.