2
\$\begingroup\$

I am storing one material ID per vertex in my meshes. In the shader for the mesh, I have a texture array full of textures that the material ID can refer to. The fragment color is chosen using triplanar texturing. Unlike vertex colors, these material IDs cannot be blended.

For example, if a dirt texture had a material ID of 5, and a grass texture had a material ID of 1, fragments in between one vertex with material ID 5 and one with material ID 1 would potentially have a material ID between 2 and 4, which is not ideal. Another flaw with this method is the fact that there would be a hard transition in between materials, as the mix or lerp function in the shader is not used.

Is there a way to blend the textures of two neighboring vertices in the fragment shader? I can add additional per-vertex data if needed.

\$\endgroup\$
3
\$\begingroup\$

One way that this can be done is to divide your materials into layer slots. As one example, REDengine 3, powering the Witcher 3, uses a background layer and an overlay layer for its terrain materials.

Slide from the CD Projekt RED GDC talk linked above

You can store your material data in control map textures as in the case above, or in your vertex data. Either way, you end up storing the identity of the material in each slot, and a blending weight between them. This keeps the total amount of material data per point and total number of texture sampling operations tightly controlled: even if your library of materials is huge, only a fixed number of them are ever in play for a single triangle/fragment.

If storing this data in the vertices, then this means that any three vertices sharing a triangle need to agree on the materials in each slot. If a vertex sits at the junction of 6 triangles, 3 of which want "snow over rock," and the other 3 want "snow over dirt," then you can duplicate this vertex to handle each case - similar to how we create hard edges where a normal changes abruptly, or UV seams. For a control map, you can use point filtering for the material IDs so they never get interpolated.

This does place a constraint on how your materials are painted. In a 2-layer system, if two adjacent locations disagree about what material should go on layer 1, layer 2 needs to hit full opacity along their shared edge to prevent a texture seam being visible. Similarly for two locations disagreeing about what belongs on layer 2 - it has to hit zero opacity in between. In practice you can usually spread out material changes over a few triangles/cells to account for this, or stick a prop/some grass in the way so the seam isn't visible. ;)

\$\endgroup\$
0
\$\begingroup\$

The vertex value will indeed be interpolated when arriving to the fragment shader, so for example having two connected vertices, one with texture slot id=2 and one with id=17 will give the fragment shader everything in between 2 and 17 (including decimal values in between) depending on pixel position, causing lots of texture slots to be walked through on the way.

A quick though might be to actually describe the amount of each texture slot in each vertex, using a 0...1 style, ie. doing it "in parallel", in an order that is always fixed. A vertex typically connects to 6 other vertices (but can connect to more), so at most 6 textures would in that case be involved in the blending at any time. You'd however describe the amount of all textures in each vertex; those that are 0 you'd simply skip in the fragment shader and maybe count on doing at most 6 lookups.

A question that comes next is how many "floating point" channels can you allocate in the vertex data? Not too many, i guess, but at least a few, depending on what else you need to bake in the vertex data.

Having said that, i could point out that you maybe do not need a fine resolution of 1/256 for the "amount of each texture"? If you are willing to have for example 1/16 as resolution, you can bake in the texture amounts for 16 different textures in one 8-bit "float-part", since it can hold 256 different integer values. In that case, you could describe the blending of max 16 texture materials in for example normal.w. Or have 8 channels, each with a resolution of 1/32. Ofc you need to do some bit maths in the shader (and when building the mesh), but that's the fun part right? :-)

Edit:

About the segment above, i must admit a duration-of-a-shower amount of brainwork did not reveal any good solutions as to how to control the splitting of a single float so that one is able to grab individual segments of it's bit-sequence while GPU is interpolating it. This would be a question for the bit-guys, ie. a separate issue to ask the encryption and compression experts. Can you interpolate a 32-bit floating point value AAAA (4 bytes), so that individual bytes of it's bit representation remain decodeable along the interpolation and all become BBBB separately at the other end?

\$\endgroup\$
  • \$\begingroup\$ The problem with this solution is that it requires many extra channels of per-vertex data. Even if I did figure out how to retrieve 4 interpolated lower-precision floats out of one 32-bit float, I would need a lot of different channels to support high numbers of materials. Also, as far as I know, any given fragment (which is where the blending is done, not the vertex shader) will have a maximum of 3 neighboring vertices, so there is a maximum of 3 materials to be blended in each triangle, not 6 like you were saying. \$\endgroup\$ – epitaque Mar 2 '17 at 22:26
  • \$\begingroup\$ 3 if the vertex touches only one other triangle, correct. But that wuold be an edge case, literally. I was considering a normal case that must be supported anyway, for example an ordinary ground tesselation, and i believe you also mostly encounter 6. Yes, the interpolation most certainly happens "outside" the vertex shader, as it only delivers values per vertex. I can't unfortunately think of any other solution to your problem, without invoking a supporting texture. Is that option out of question? \$\endgroup\$ – Stormwind Mar 2 '17 at 23:11
  • \$\begingroup\$ A supporting texture is definitely not out of the question, as long as it isn't really expensive. \$\endgroup\$ – epitaque Mar 4 '17 at 1:04
  • \$\begingroup\$ Well imagine you have a homogenous tesselation of 1025x1025 vertices and a corresponding 1024x1024 texture holding material indexes. Each vertex has U and V growing with 1/1024 linearly. In the fragment shader, at any point in a face, you'll be able to resolve a set of 2x2 or 3x3 nearby pixels to lookup (point sampling!) - say you eventually do max 4 lookups at any time, ie. select the 4 most relevant pixels and read a material index from each. Then do a blending based on distance to vertex (= to "1/1024" UV). The Q is can you wrap this tesselation to your object? \$\endgroup\$ – Stormwind Mar 4 '17 at 14:58
  • \$\begingroup\$ Several issues with this idea. (1) Since the vertices aren't on a flat plane, this would require a 3D texture of material indices. (2) You would have to find the distance to the nearest material index in 3 axes, which might get expensive. (3) You would still have somewhat hard transitions because with a coarse distance calculation, many fragments will have the same calculated distance. (4) It might be difficult to do chunking with this solution (neighboring chunks would not be able to sample neighboring material ID textures) (5) Much of the data in the 3D texture would not be used \$\endgroup\$ – epitaque Mar 4 '17 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.