Game Development Stack Exchange is a question and answer site for professional and independent game developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

As a learning experience, I'm developing my own 3D game engine using OpenGL. I'm a little confused as to how to implement my rendering engine such that it uses a programmable pipeline while still being responsible for projecting and displaying vertices correctly.

Let's say my engine has a simple vertex shader that uses projection and modelview matrices to project a vertex in 3D space. How do I use the engine's vertex shader while still allowing the programmer to provide his/her own vertex shader for other calculations in their game? What is the normal approach here?

share|improve this question
I think the usual approach is that if the user is writing the shaders, it's his/her responsibility to write them correctly. The engine can provide standard shaders as examples and as a basis for user modification, but it shouldn't try to enforce that the shader uses the data in the "expected" way. On the contrary - a big reason for users to write their own shaders is so they have the freedom to do things differently if they desire. – Nathan Reed May 20 '13 at 0:20
If you use deferred rendering, you could hard code the g-buffer pass. Then programmers using your engine could write their own shaders in GLSL and get textures from the g-buffer and from prior passes as samplers. Related question:… – danijar May 20 '13 at 0:27
up vote 3 down vote accepted

How do I use the engine's vertex shader while still allowing the programmer to provide his/her own vertex shader for other calculations in their game? What is the normal approach here?

There is no normal approach because engines generally don't let you do that.

Generally speaking, engines take one of two approaches: either the user provides none of a shader or all of a shader. Never parts of a shader.

Engines that allow the user to provide shaders will establish some form of convention for communication between the application and the shader. They'll say that the position must be named "position" or whatever, and you're expected to provide vertex shaders with that attribute name. The various transformation matrices will similarly have specific uniform names and types, and the user providing the shader must adhere to those names and types.

Some engines allow you to create shaders from raw components via some system. But even there, it's the system that's creating those shaders. If that system has some mechanism for inserting user-provided fragments of code, then it will be specific to that system, and it likely won't be in GLSL anymore. And even then, those fragments will have to adhere to a specific convention based on the needs of that system.

In any case, if you're truly writing an engine for yourself or a single team that you're supporting, you really shouldn't bother with this. Just use shaders directly.

share|improve this answer
Good answer, thank you! – Zanneth May 21 '13 at 1:10

Higher-end engines typically use a higher-level abstraction than an individual shader.

Simpler games just allow each model/material to specify shaders and then to use HLSL includes or the like to allow all common code to be easily reused. You'll need a bit of discipline in applying the shader includes properly and it can be a bit repetitive sometimes, but so long as you don't have a really huge number of different materials (most hobby/student games won't), you should be okay.

The higher-level abstraction are generally built around a concept of shader pieces that can be assembled into complete shaders. For example, the engine would provide some shaders for deferred shading vs simple pass-thru. A material could specify a BRDF which maps to a small piece of shader code and metadata. The engine/tools look at the BRDF fragment and inserts the necessary code into the pixel shader to do the calculations and inserts the correct attribute code into the vertex shader, binding the final generated shaders to the runtime material definitions. This is non-trivial to implement for someone just beginning at games.

Some engines go so far as to use a visual material definition language and generating the shaders from that. This is a lot of work, though.

share|improve this answer
Indeed, inventing my own language for inserting BRDF code into a shader is something I'll avoid for my first game project. ;) Thanks! – Zanneth May 21 '13 at 1:12

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.