How do you store uniform data?

Question

In a general purpose C++ rendering engine, one way of organizing data is to divide mesh data into two classes: the geometry and the material. The geometry class includes indices, vertices, normals, and texture coordinates. The material class, as established by this answer, is often used to store the shader and uniforms.

Uniforms can be many types, such as an int or a float. So you can't just have one member variable in your material class defined std::vector<GLint> uniforms, as in that case you would only be able to add uniforms of integer type. There are two solutions that I know of:

1. Adding a vector for each type of data you want to use
   In this case the material class would look like the following:

   class Material {
     ....
     std::vector<GLint> intUniforms;
     std::vector<GLfloat> floatUniforms;
   }
   

The problem with this is that it is very verbose and would probably require a lot of different functions (for each type) to add and delete uniforms.

2. Creating a class called Uniform, having a member variable of type any.

   class Uniform {
     UniformType type;
     GLuint size;
     any value; // This will be a float/vec2/vec3 ...
   }
   
   class Material {
     std::vector<Uniform> uniforms;
   }
   

However, using the 'any' class feels cheaty and seems to break the static typing design pattern used in C++.

My question was, is there another method of storing variable uniform data that supports a set of types without being verbose or using the any class? Or is there a design flaw with this approach and I should organize the Geometry/Shader/Uniform data differently?

Answer 1

Neither of your solutions are ideal.

First, realize that the glUniform* calls are mostly a bad idea. Don't use them. Use Uniform Buffers instead. The buffers offer several primary benefits.

One benefit is that you can separate out big blocks of "constant" uniforms (any per-model or per-instance data) into its own uniform buffer and then only need to bind that buffer rather than needing to bind many individual uniforms. Another is that the buffer binding can also be shared across many shaders if necessary. Third, you can bulk upload related sets of uniform data.

In general, uniform buffer objects are much closer to how the actual hardware works, and will results in faster graphics pipelines. That's not a hard rule, mind, but a decent guideline; you're best off testing and measuring, of course.

Once you move to uniform buffers you might find yourself facing the same problem though slightly rephrased: what uniforms exist in your buffer, what are their types, and what are their offsets?

There are two main solutions here. The first is to standardize your buffers. Different shaders should use an identical layout for, say, your per-frame camera information (eye position, screen dimensions, projection matrices, etc.). Then your CPU-side code doesn't need to "detect" anything or handle different layouts. It just prepares the per-frame uniform buffer in a single hard-coded way and all the shader consume that buffer is a single ahem uniform way (use a shader preprocessor to just #include shader code that declares the uniform buffer, its binding location, and its layout properties). If you change the shader code, change the C++ code to match.

That technique can take you surprisingly far. You're likely to find that the vast majority of your shaders use the same sets of inputs from the native code. Hardcode those and enforce that shaders consume the data in the format your CPU code prepares. This essentially reverses your original problem: instead of CPU code being dependent on shader layouts, the shader layout is dependent on CPU code. The shader preprocessor keeps the code in sync. (Depending on the shader language, you might even be able to share the exact same .h file between your shader and your CPU code.)

For content-driven shaders with per-model uniform attributes, the above may or may not work. In these cases, though, you might note two things: (1) the runtime code never modifies these values so the buffer contents only matter to the shader and the editor, and (2) since these values are per-model you probably want them in a separate buffer from your per-instance or per-frame values so you can make minimal rebinds and uploads.

The editor doesn't need to be particularly speedy. You can store your uniforms in whatever format you want, so long as you bake them down into the proper bag of bytes for export to the shader. One approach here is to use metadata to describe the uniform values of the model, something like:

struct ShaderUniformMetadata {
  char* data = nullptr;
  EType* uniformTypes = nullptr;
  uint16_t* uniformOffsets = 0;
  uint16_t size = 0;
  uint16_t uniformCount = 0;
};

The metadata contains information about the available uniforms and their types. To read/write a uniform, memcpy from/to data + uniformOffsets[index], using the value in uniformTypes[index] to know the correct size.

To calculate the size of the buffer, sum up the appropriate sizes/alignments from the uniforms. An algorithm can pack the values appropriately to get the minimal size while maintaining the necessary alignment for each input.

To actually create the buffer at runtime, just load whole blob from your game data file and then pass the size and data values to glBufferStorage. Now you just bind that buffer before rendering that model.

Your per-instance buffers are the hard-coded layouts so there's not a lot of special magic to do there.

Answer 2

If you want to avoid storing a 'type' variable, then polymorphism is maybe what you're looking for.

An example:

struct Uniform 
{
  int id; 
  virtual void Send() = 0; 
};

struct IntUniform : Uniform
{
  int value; 
  void Send() 
  {
    glUniform1i(id, value);
  }
};

struct FloatUniform : Uniform
{
  float value; 
  void Send() 
  {
    glUniform1f(id, value);
  }
};

struct Material
{
   std::vector<Uniform*> uniforms; 
};

If you're willing to give up genericity for performance, here's how an engine I wrote works.

int activeShader; 

struct Shader 
{
 ... methods to create & bind shader
};

struct ShaderUniform
{
    Uint id;
    ShaderUniform() {}
    void Link(const char* identifier)
    {
        id = glGetUniformLocation(activeShader, identifier);
    }
    void operator<<(float value) {
        glUniform1f(id, value);
    }
    void operator<<(Vector4 value) {
        glUniform4fv(id, 1, &value[0]);
    }
    void operator<<(Vector3 value) {
        glUniform3fv(id, 1, &value[0]);
    }
    void operator<<(Vector2 value) {
        glUniform2fv(id, 1, &value[0]);
    }
    void operator<<(Matrix4 value)
    {
        glUniformMatrix4fv(id, 1, GL_FALSE, &value[0][0]);
    }
    (...) 
};

struct : Shader 
{
    ShaderUniform cameraMatrix, wind(...) 

    void Initialise() 
    {
       cameraMatrix.Link("cameraMatrix"); 
       wind.Link("wind"); 
       (...) 
    }
} MyShaderABC; 

Usage:

MyShaderABC.Use(); 
for (int i=0;i<objectsOfSomeKind.quantity;i++)
{
   Matrix4 someMatrix; 
   float someFloat; 

   MyShaderABC.cameraMatrix << someMatrix;
   MyShaderABC.someVariable << someFloat; 
}

Note that there is no security, you can send the float into the matrix uniform, but if you're the one using the engine, that's not a problem.