Noisy edges, smoothing out edges between faces via fragment shader

Question

I have a generated terrain, with hexagonal geometry, as per screenshot below:

I then generate biomes, but as you can see the borders between them are really ugly and straight. To hide that hexagonal origin, I would need to smooth out the borders between biomes. This is how it looks now in wireframe with real tringular faces:

What I'm aiming for is something more like this:

Each vertex has attibute that holds the biome type, I can also add special attributes to the vertices on the edge between two biomes, but I just don't seem to be able to figure out how to pull this off in shader code, obviously noise is involved here, but how do I make it continous across multiple faces and entire border of multiple biomes?

I'm rendering with WebGL using THREE.js

Answer 1

Other answers here suggest using a texture. Here's a technique that doesn't use textures.

You want the boundaries between hexagons to be interesting. It's easier to make interesting boundaries when you move them into the center of what you're drawing. Instead of drawing the tiles directly, you draw the “dual” of the tile. This technique is called “corner tiles” (here and here and here). The dual of a hexagon is a triangle, so we would draw these triangles instead of the hexagons:

The boundaries between hexagons are now in the middle of the rendered triangles, so that'll let us do more interesting things with them. Bonus: you only need to draw two triangles per hexagon, instead of six (or twenty four as you are doing now).

Inside each of those triangles we want the fragment shader to draw the hexagons. We can do that with barycentric coordinates. Put (1,0,0), (0,1,0), and (0,0,1) at each vertex of the triangle. Inside the triangle, those coordinates will be interpolated. The fragment shader will receive (a,b,c) and can look to see which value is largest — that will tell us which of the three hexagons should be drawn at this point.

float max_n = max(barycentric.r, max(barycentric.g, barycentric.b)); if (max_n == barycentric.r) { color = v_color0; } else if (max_n == barycentric.g) { color = v_color1; } else if (max_n == barycentric.b) { color = v_color2; }

That's for straight lines.

If you want noisy edges, you can add noise to the barycentric coordinates:

By playing with the amplitude wavelength/frequency of noise, you can get some cool effects:

You need to be careful with the noise, making sure it's consistent across triangle boundaries. One way to do that is to pass in a hex id and use that as the seed value for each of the three noise values added to the barycentric coordinates.

I made an interactive demo here. (For the demo I didn't implement the hex id or some of the other things you might need if you were making this work in a real project — it's just a quick & dirty demo)

Answer 2

I'm sure that "could" be solved with some image algorithm but if it was me I'd probably solve it with textures. I'd make hexagonal textures, put them all in a texture atlas, then for each hexagon I'd look at its neighbors and decide which texture to apply.

The textures would need to have versions for each type of terrain plus versions for each type of transition.

This is similar to how many tile based systems do terrain. Here's an example from 2d games.

Another possibility would be to just have your textures for your various types of terrain (water, snow, dirt, grass) and add mix amounts to each vertex of hexagon to decide how to mix them.

This article shows the idea of mixing terrain textures. I'm not suggesting following their implementation but it shows the idea.

Answer 3

First, render your biomes to a texture. Map the triangles to texcoords. You can do this using a mercator projection, or, better, a cube map. Now, in the fragment shader, do something like this:

// frequency and magnitude of the noise in texels.
uniform float frequency;
uniform float magnitude;

// This function should just look like random smooth noise in x,y
// This one isn't great, but you can experiment.
vec2 noise(vec3 vertexPos) {
    return vec2(sin(vertexPos.x * frequency + vertexPos.y * frequency) * magnitude, 
                cos(vertexPos.y * frequency * 2 + vertexPos.z * frequency) * magnitude);
}

// Sample the texture and preturb it with noise based on the world
// position of the fragment.
vec4 frag(vec2 texCoord, vec3 fragmentPos) {
   return texture(mySampler, texCoord + noise(fragmentPos));
}

where noise is some pseudo-random function (using, say, sinusoids) on the 3D position of the vertex in model space which returns a noisy offset to the texture coordinate. Sample the texture using GL_NEAREST to keep sharp borders.