1
\$\begingroup\$

I'm attempting to render grass using GLSL tessellation and geometry shaders. For vertex input, I'm using position and normal. For reference, here's an image of the grassy field without tessellation (just vertex, geometry, and fragment shaders). The geometry shader transforms individual points on the underlying mesh to rotated quads.

Non-tesselated grass using vertex, geometry, and fragment shaders.

So, on to the tessllation step. Here's the vertex shader I'm using:

#version 440 core

in vec3 vPosition;
in vec3 vNormal;

out vec3 hPosition;
out vec3 hNormal;

void main()
{
    hPosition = vPosition;
    hNormal = vNormal;
}

Next up is the tessellation control (TC) shader. I'm passing position and normal through unchanged. Tessellation levels are hardcoded for testing purposes.

#version 440 core

layout (vertices = 1) out;

in vec3 hPosition[];
in vec3 hNormal[];

out vec3 ePosition[];
out vec3 eNormal[];

void main()
{
    ePosition[gl_InvocationID] = hPosition[gl_InvocationID];
    eNormal[gl_InvocationID] = hNormal[gl_InvocationID];

    if (gl_InvocationID == 0)
    {
        gl_TessLevelOuter[0] = 3;
        gl_TessLevelOuter[1] = 3;
        gl_TessLevelOuter[2] = 3;
        gl_TessLevelInner[0] = 3;
    }
}

The next stage is tessellation evaluation (TE). Here's my shader:

#version 440 core

layout (triangles, equal_spacing, cw) in;

in vec3 ePosition[];
in vec3 eNormal[];

out vec3 gPosition;
out vec3 gNormal;

void main()
{
    gPosition = tessLerp(ePosition[0], ePosition[1], ePosition[2]);
    gNormal = normalize(tessLerp(eNormal[0], eNormal[1], eNormal[2]));
}

Where tessLerp is defined as follows (applying barycentric coordinates):

vec3 tessLerp(vec3 v0, vec3 v1, vec3 v2)
{
    return v0 * gl_TessCoord.x + v1 * gl_TessCoord.y + v2 * gl_TessCoord.z;
}

My goal, then, is to have the output of the TE shader feed into the geometry shader, at which point the geometry and fragment shaders are unchanged from the non-tessellated version above. As input, the geometry shader takes layout (points) in;. Instead, I'm getting nothing (the curved green mesh itself is rendered separately from the grass):

Grass isn't rendering when using tessellation (with the same vertex, geometry, and fragment shaders).

Some common answers I've found in similar questions that I'm confident are not the problem here:

  • I'm calling glPatchParameteri(GL_PATCH_VERTICES, 3); before rendering.
  • My draw call is already using GL_PATCHES rather than GL_POINTS (used for the non-tessellated version). The specific draw call is glDrawElementsBaseVertex(GL_PATCHES, indexCount, GL_UNSIGNED_INT, (void*)0, 0);, where indexCount is the number of indices bound in the element array buffer for the mesh.
  • In the TC shader, I've tried layout (vertices = 3) out; rather than 1. My understanding is that using 3 causes indices to be computed (in order to form triangles), which I don't need for my purposes (since each tuft of grass is built from a single point).
  • From the vertex shader through the TC and TE, vertex positions are left unmodified from the base mesh (rather than being transformed to clip space). Instead, it's the geometry shader that applies the view-projection matrix when emitting four vertices to form a quad.

My understanding is that by rendering in this way (indexed) and with GL_PATCH_VERTICES set to 3, OpenGL will group every three vertices (i.e. each triangle) into a patch. Each triangle's data (three vertices, each containing position and normal) is passed from the TC shader to the TE shader (with primitive generation in between), then the TE shader computes new data (position and normal) by multiplying each tessellated vertex against barycentric coordinates. This results in a series of vertices that are passed to the geometry shader. Clearly, either my understanding is wrong or I've configured something incorrectly.

Why is my tessellated output not reaching my geometry shader?

\$\endgroup\$
0
\$\begingroup\$

I solved it!

  1. Rather than layout (triangles, equal_spacing, cw) in;, I used layout (triangles, equal_spacing, cw, point_mode) in; (note the fourth point_mode parameter). See the Khronos documentation at https://www.khronos.org/opengl/wiki/Tessellation_Evaluation_Shader (specifically sections 1.3 and 1.4 regarding primitive ordering and primitive generation).

  2. By itself, this caused the grass to reappear, but only the first entry in the TE shader's input arrays was populated. This resulted in interpolated grass tuft positions (using the tessLerp function) to lerp from the origin, as pictured below (the origin in this screenshot is the top-center of the central pillar):

Incorrect tessellation from the origin.

To fix this, I changed layout (vertices = 1) out; in the TC shader to layout (vertices = 3) out;, resulting in proper interpolation.

Corrected tessellation after modifying the TC shader.

So, to summarize, the reason my tessellation output wasn't reaching my geometry shader seems to have been that my geometry shader was expecting points, but the TE shader was outputting triangles.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.