I'm working mostly with three.js and webgl, i have some experience with unity.
I don't consider three.js a game engine, more of a "rendering library" - something that takes care of the GL boilerplate, but is still flexible enough and lets you build on top of it.
As such, i believe it lacks some optimizations that a game engine would have, and some things are even the opposite - sacrificing optimization for convenience.
For example, there isn't a specific "uploadDataToGPU" method on a mesh/geometry, rather the library will make a gl_bufferData() call when it encounters a new geometry within the render loop.
I've encountered a pull request that shook up my understanding of how graphic libraries work so i'm hoping to clarify some things here :/
Drawcall batching
I know that there is overhead involved in issuing a draw call. But i'm not sure where the tradeoff is when grouping draw calls, or merging them all together.
When i have a camera, it's two uniform matrices, shared by all shaders.
Can this get set once per frame, and webgl/opengl won't have to bind this uniform between different draw calls? I'm not actually sure how this depends on the shader, and if it can always be the same location even though the shaders are different but are using the same camera:
gl_uniformMatrix4fv( commonShaderChunkWithCamera , "u_cameraProjectionMatrix" );?
This is from a webgl demo and i'm not sure what kind of optimizations it has in place. It's drawing the same mesh it seems, these draw calls are sequenced, (the same mesh is being draw one after the other) but the calls look different:
I'm curious if i'm interpreting the stuff i've highlighted right:
A high-level scene graph has described that there are N nodes containing geometry M. This geometry got uploaded automagically to the gpu, once. The buffer exists and can be bound for drawing. The high-level graph also described a high-level material, and a TRS matrix trough abstract properties such as "position, rotation, scale".
I can guess that the material is most likely the same, not just on a shader level, but that it probably binds the same texture, color, glossiness, etc. since none of those get changed in the draw call. What does seem to differ are these three matrices, which i would guess are modelMatrix(TRS), normalMatrix(3x3, derived from TRS), and possibly modelViewMatrix (which may be redundant if there's a modelMatrix, and a viewMatrix, but i know both are available, so my guess is that it gets computed per node from the two).
Can this further translate to:
"in between two drawcalls of the same thing, with the same material, the renderer figured out to only update the portion of the shader that deals with moving the object"?`
and/or
"The renderer figured out how to sequence the draw calls so that the same geometry, with the same material, different TRS get's drawn in a group, so that there is minimal state change involved".
I'm not sure what could be going on in between the two highlighted calls, since it seems like the same mesh.
sorting
I'm a bit confused when it comes to sorting, overdraw and early depth testing. I'm not quite sure why the fragment shader has to run if we get the depth information at the end of the vertex transformation, but when i read about the early testing it sounded a bit like deferred shading. The article mentioned running just the vertex shader with the most simple fragment shader and writing only to the depth buffer. Still, having opaque objects depth sorted would lead to the most possible GL state changes in the worst case scenario (each consecutive draw call changing the program, geometry etc)?
So, does depth sorting only matter for transparency, while for opaque objects sorting means "group objects in such a way that there is minimal GL state change happening"?
Texture lookup vs attributes when instancing
The proposal for batching uses instanced arrays, but instead of using attributes, it stores the TRS and matrix normals in a texture, unpacking them like so:
mat4 getModelMatrix() {
float dx = 1.0 / float( instancingTextureSize );
float dy = 1.0 / float( instancingTextureSize );
float j = instanceIndex * 4.0;
float x = mod( j, float( instancingTextureSize ) );
float y = floor( j * dy );
y = dy * ( y + 0.5 );
vec4 v1 = texture2D( instancingTexture, vec2( dx * ( x + 0.5 ), y ) );
vec4 v2 = texture2D( instancingTexture, vec2( dx * ( x + 1.5 ), y ) );
vec4 v3 = texture2D( instancingTexture, vec2( dx * ( x + 2.5 ), y ) );
vec4 v4 = texture2D( instancingTexture, vec2( dx * ( x + 3.5 ), y ) );
return mat4( v1, v2, v3, v4 );
}
The argument is that too many attributes are already being used for morph targets, skinning etc, so that this matrix (along with the normal matrix) can be stored in a texture, or even a uniform.
This doesn't seem right, but i'm not sure if i understand things correctly:
- size. Uniforms have their own limits. With attributes, i could draw many objects at once, not sure of limitations, but is say 2^16 a valid number? Uniforms can store far less data? Same goes with textures - either it complicates the lookup logic even more, or is limited to WIDTH or HEIGHT?
- instructions. It takes a few instructions to do a fetch from a texture, finding these strides, texel size etc. I've read that back in the day it was cheaper to do a cubemap lookup than it was to normalize a vector. I see that some of these divisions could go away in favor of multiplication, still though, unpacking all this 7 times seems like a bit of overhead? With attributes one would just fetch these values.
- texture lookups. I don't understand the cost of doing these in this context. If a number of draw calls are going to be eliminated, what is the tradeoff between using an attribute, and doing the texture fetch, say if there were none of this unpacking logic going on. Is it safe to say that 1:1, something that does 4 texture fetches, instead of using a
uniform mat4is going to be slower?
instancing candidates
What even benefits from instancing? The proposal is sort of to just instance everything that shares geometry/materials.
The first thing that comes to mind is CPU culling.
One might want to be in control of how these objects are being grouped so they could still be eliminated if they're in a different room, behind the camera etc. Is this a valid way of thinking? One would have to be aware of the context, and make a decision based on that?
- do these objects exist in different rooms
- are these objects laid out in some kind of a grid, can be culled by zones
- are these objects extremely heavy, could in this case saving draw calls be insignificant compared to culling them aggressively on the cpu? But even here, if it's a cluster of gargoyles, it might make sense to batch, but it wouldn't if they're spread out in the game world.
Unity seems to want the user to explicitly designate an object as "static" in order to be a candidate for this type of optimization.
I'm curious how this applies to a design decision "lets save attributes for morph targets and skinning and use textures". Would one want to have a skinned/morphable mesh instanced?
What kind of behavior should one expect from a library such as three.js?
I imagine because it's javascript, users struggle with draw call overhead even more than they should. Not necessarily the GL part, but transforming all the matrices, and doing all the math on 64bit floats, along with garbage collection is what causes most problems.
Still though, the smarter it would get, the harder it would be to build something more engine like on top of it? This is kinda how i see the library, makes it much easier to work with GL, while still working on just the drawing part.
