What are the common rendering optimization techniques for the geometry pass in a deferred shading renderer? [closed]

I have been developing a game engine using OpenGL 3 and C++ (and glfw for window management). I have advanced so far, got most of the things done except sound entities and optimizations. The engine uses deferred shading so since deferred shading is itself a tiring process for an average GPU, I want to optimize the rendering process as much as possible.

The current system consists of a Scene, containing a Renderer and the current World and the World holds entities and lighting entities as separate std::vectors.

So basically every time the Scene gets called by ->render(), and it calls the Renderer, passinng the world as a parameter and gets the entity iterators from the world, draws them to the FBO and then goes through the lighting entities for the second pass. And I think this is not enough.

My current algorithm iterates through everything even if the entity is not in the screen space. I am thinking of a way to optimize the current rendering algorithm so it only calls the API functions only for the visible objects, so what are the common techniques for optimizing such renderer ?

closed as too broad by Sean Middleditch, bummzack, Josh♦Nov 26 '13 at 16:15

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

Deferred shading is only a technique to "defer" the actual shading operation for later stages, this can be great to reduce the number of passes needed (for example) to render 10 lights which needs 10 passes. My point is regardless of the rendering technique you are using there are certain possible rendering optimizations that reduce the number of objects (vertices, normals etc) that your rendering pipeline need to process.

There are no de facto standard for rendering optimizations, but rather a number of techniques that can be used interchangeably or together to achieve certain performance characteristics. Using each technique highly depends on the nature of the scene being rendered.

Deferred rendering tries to solve the problem when the number of lights increases, which in forward rendering might make the number of passes explode.

Those techniques does not directly optimize the deferred shading part, but according to your description, the deferred shading part is NOT your problem. Your problem though is that you are submitting the whole scene to rendering pipeline. So your engine has to process (for example all the 100 million vertices) in your scene just to be able to submit the result to the g-buffer, while most of theses 100 million vertex can trivially be culled away, and not submitted to the pre-process vertex and fragments pass.

In case of a forward renderer the N vertex will be processed by the vertex stage as a total of vertex count*lights count and by the fragment stage as a total of fragments count*number Lights, deferred shading effectively reduces this to only vertex count for the vertex stage and fragments count for the fragment count, before resolving the actual shading. But still N can be too much to process, especially when most of them can be trivially culled.

This makes culling more effective in case of forward rendering/multiple passes. But keep in mind that most engines will use a dual rendering approach, because deferred shading alone can not resolve transparent objects, this makes using those optimizations a must, I don't know of any commercial engine that don't do all of them.

Frustum Culling

Only Objects that are fully or partially included in the view frustum, ever need to be submitted to the rendering pipeline. This is the basic concept of frustum culling, unfortunately checking if a mesh is in/out of the view frustum can be an expensive operation, so instead, engine designers use an approximate bounding volume like an (AABB)Axis Aligned bounding box or bounding sphere, even though this might not be as accurate as using the actual mesh, the accuracy difference isn't worth the trouble of checking with the actual mesh.

Even with bounding volumes, you don't really need to check each one, alternatively you can construct a bounding volume hierarchy to do an earlier culling, using this is highly dependent on the complexity of the scene.

This is a good and a simple technique for a smaller engine, and is almost used in every engine I ever used. I recommend using a "normal" Bounding Volume/Frustum checking without hierarchies if your engine does not require rendering very complex scenes.

Back face culling

This is a must, why draw faces that won't be visible anyway? rendering APIs provide an interface to turn on/off back face culling. Unless you have a strong reason why not to turn it on, like some of the CAD applications that need to draw backfaces in certain circumstances, this is a must do thing.

Occlusion Culling

Using the Z-buffer you can resolve visibility determination. But the problem is that Z-buffer isn't always great in terms of performance, since Z-buffer can only be resolved at later stages of the pipeline, objects being occluded should be rasterized and might be written to the Z-buffer and the Color buffer before failing the Z test.

Occlusion culling solves this, by doing some early tests to cull occluded objects that are in the rendering frustum. One practical implementation of occlusion culling is using point-based queries and checking if certain objects are visible from a specific point view. This can also be used to cull lights that do not contribute to the final image this is especially useful in a deferred engine renderer.

A great real world example of such technique is in GTA5, where the skyscrapers are stratigically placed at the center of the city, are not only decorations, but they also work as occluders, effectively occluding the rest part of the city and preventing it from being rasterized.

Level of Detail

Level of detail is widely used technique, the idea is to use a simpler version of the mesh when the mesh is less contributing to the scene. there are two common implementations; one simply switches the mesh with a simpler one when it's no longer greatly contributing, the mesh is selected based on some factor like the distance and the number of pixels (area on the scree) the mesh is occupying. The other version dynamically tessellates the mesh, this is widely used in terrain rendering.

What if all of these didn't work ?

Well, that's a good question.

The first thing you need to do is to Profile your application using a graphics profiler, and determine where the bottleneck is. Keep in mind that the bottleneck may change as the content being rendered change. Bottlenecks might be also be part of the code running on CPU so you need to measure that too.

After that you need to do some optimizations on the bottleneck, keep in mind that there is no right answer for this, and will be different from hardware to another.

Some common GPU optimization tricks:

• Try different vertex structures for example {VNT} interleaved in the same array or {V},{N},{T} in different arrays.
• Draw scene front to back.
• Turn off Z-buffer at some points for example if an image doesn't need Z testing.
• Use compressed textures.

Some common CPU optimization tricks:

• Use inline functions for small functions.
• Use SIMD (Single instruction multiple data) when possible.
• Avoid cache unfriendly memory jumps.
• Use VBOs with the "right" amount of data. (depending on your hardware) but usually less draw calls are better.

But what if my bottleneck was in the deferred shading ?

In this case, since deferred shading is more concerned about lights then the most obvious part is to optimize the actual shading calculations. some of the points to keep an eye on:

• Render lights that actually affect the final image. In other words cull the lights that don't contribute. This can be effectively implemented using the occlusion culling I mentioned before.
• Does this light need the specular or some other components? Maybe not.
• Does this light cast shadow ? Some lights don't need to cast shadows.
• Can this light contribution be pre-computed? If it is not moving probably some aspects can be pre-computed.
• Sorry, these have nothing to do with deferred shading, they are actually the exact performance issues that the technique mitigates effectively and therefore the least useful optimizations to make, the focus should be on the lighting pass(es) because if the lighting cost is not the dominant time-taker, deferred shading is probably the wrong choice. – MickLH Nov 26 '13 at 9:38
• @MickLH Unfortunately, apparently you didn't read the question, his problem was mainly that he iterates over the whole scene everytime, and he didn't mention any bottleneck regarding the deffered shading. I mentioned at first that deffered shading solves the passes explosion problem when having a lot of lights/materials. But I then added that those are a must optimizations for any engine regardless of the shading technique forward or deffered. Regarding that these are the exact issues that the technique migitates I strongly disagree, I can't tackle the whole points here (follows) – concept3d Nov 26 '13 at 9:59
• it's really stupid to build a deferred engine without frustum culling for example, so the engine will process for example (100 million vertex) just to be able to submit the result to the g-buffer. The differed shading solves a different problem, which was not his problem, his problem was submitting all the geometry to the pipeline. – concept3d Nov 26 '13 at 10:23
• though I agree on the part that some optimization should occur on the lighting calculations, and if light calculations was not the dominant then deferred is the wrong way to go. but again that was not his problem. – concept3d Nov 26 '13 at 10:26
• I'll retract my downvote if you make it clear that these optimizations are actually the least effective for a deferred renderer, as it stands you haven't shown him/her + googlers that the performance issue has nothing to do with deferred shading. – MickLH Nov 26 '13 at 10:31

Your problem is not related to deferred shading whatsoever, you need to implement the basic core elements of a renderer before you try to speed up some specific part.

When you have finished with what concept3d has explained, if you actually find that you need to optimize the deferred shader itself (as opposed to the whole rasterization pass) you can implement Tile-Based Deferred Shading.

If you are not limited by the number of dynamic lights you should consider why you are using deferred shading at all, but if you are then you will want to try the optimization that made Battlefield 3 possible. (They hint to it in slide 10 of their public PDF: http://dice.se/wp-content/uploads/GDC11_DX11inBF3_Public.pdf)