Is this scheduling operation a slowdown for the CPU, or is it a benefit?

Question

I'm making a custom engine, and after much research, I realized that the state changes and the vertex or texture data loadings to VRAM are the most slow operations.

So I want to optimize it making a wrapper between OpenGL and a rendering engine that has a set of functions like OpenGL's, but making it like it saves the calls in arrays. After the "pre-rendering" phase, it schedules all operations, when possible, to make the state changes of GPU less frequent. Next, the scheduler groups vertices to make contiguous packs of vertices, when possible, and render it all, together.

Example of interface

class Renderer
    {
        typedef int Texture; // TMP
        class State 
        { public:
            Vertex * Start;
            Vertex * End;
        };

        class SOC : public State
        { public:
            Shader * Reference;
            // End Null if update
        };

        class EOC : public State
        { public:
            GLenum * Reference;
            Vertex * Start;
            Vertex * End;
        };

        class FOC : public State
        { public:
            FrameBuffer * Reference;
            // End Null if draw
        };

    public:

        // Rendering api
        //-------------------------------------------------
        static void End();
        static void Begin(GLenum);
        static void Enable(GLenum);
        static void Disable(GLenum);
        static void BlendMode(GLenum, GLenum);
        static void BindShader(Shader *);
        static void UnbindShader(Shader *);
        static void UpdateShader(Shader *);
        static void SamplingType(GLenum, GLenum, GLenum);
        static void DrawFrameBuffer(FrameBuffer *);
        static void BindFrameBuffer(FrameBuffer *);
        static void UnbindFrameBuffer(FrameBuffer *);
        static void Vertex2(int, int);
        static void Vertex2(float, float);
        static void Vertex2(int, int, Color);
        static void Vertex2(float, float, Color);
        static void Vertex2(int, int, Texture);
        static void Vertex2(float, float, Texture);
        static void Vertex2(int, int, Color, Texture);
        static void Vertex2(float, float, Color, Texture);
        //-------------------------------------------------

        // Rendering phases
        //-------------------------------------------------
        static void Initialize();               //   0
        static void PreparePipeline();          //   1
        static void Schedule();                 //   2
        static void StartRenderPipeline();      //   3
        static void RestorePipeline();          // (0.5)
        //-------------------------------------------------

    private:

        // Rendering source
        //-------------------------------------------------
        static Vertex           * GlobalVertexBuffer;
        static std::vector<SOC>   ShaderStates;
        static std::vector<EOC>   EnableStates;
        static std::vector<FOC>   FBOStates;
        //-------------------------------------------------

        // Handlers openGL
        //-------------------------------------------------
        static GLuint StreamVertexBuffer;
        static GLuint Index;
        //-------------------------------------------------

        // Current state
        //--------------------------------------------------------------
        static std::unordered_map<GLenum, bool>         ActiveFlags;
        static std::unordered_map<Shader *, bool>       ActiveShaders;
        static std::unordered_map<VertexBuffer *, bool> ActiveVBOS;

        static GLenum CurrentPrimitive;

Is this scheduling operation a slowdown for the CPU, or is it a benefit?

Answer 1

As far as I know, modern high performance rendering engines gather render operations into a so called "render queue".

Each such operation is basically a single drawcall: it has buffers, shaders, shader uniform data, textures, whatever other state needed...

These are then sorted, most expensive to switch property first. Shaders are expensive to switch, so maybe sort by those first, then within each chunk of render ops with same shader, sort by the next expensive thing (lets say, by texture) (this is just an example, you want to figure out the best sorting method through profiling and maybe researh of existing solutions)

One way to do that sorting is pack all the data into an integer key (say a 64 bit int) with the keys of the expensive stuff stored in the most significant bits, and keys of less expensive stuff in the other end. This allows for fast sorting by this single key. Same as if you were sorting based on numbers between 0 and 999, and let shader config key be the hundreds, texture config key be the tens, and buffer config key the ones.

There can be a higher level ordering of render stages/passes as well - eg you often want to render all transparent geometry after the opaque geometry, for example.

Thats for the expensive state changes.

For the less expensive changes, sorting would likely not be worth it, so instead you can just maintain a cache of the current state and always check if the state is already at the desired value before setting it. If you dont go with the renderqueue approach (which takes effort to implement) this is definitely better than having nothing at all.

For further research into the specifics, you can search using the phrase "Implementing a render queue" perhaps combined with "OpenGL" (although results for DirectX should apply to OpenGL pretty well too)

To answer the question, yes the sorting will of course take some CPU time (choose a sorting algorithm with good computational complexity in cases where data is already nearly sorted - so assuming you keep the sorted data from previous frame and make the few changes that happened for current frame, sorting should be quick) - if you are bottlenecked by GPU this should not be an issue (its not THAT expensive). Also consider that by avoiding redundant state changes, you avoid any CPU side processing done by drivers (even if they did nothing, there'd still be a trip to the driver), so its likely a win for CPU as well.

Answer 2

I agree with everything @Waterlimon stated, that's how our custom game engine works: traverse the scene graph, collect every item to be rendered into either an opaque or transparent render queue (each entry is a pointer to a mesh plus the current world matrix for that mesh and any other render values calculated during the scene graph traversal, like opacity), then sort the transparent queue by depth (you could also sort the opaque queue by shader, however we don't bother to do this).

Regarding batching, we recently added a batch manager which enabled us to achieve PlayStation 3 performance on an iPhone 4. On mobile devices, especially of that generation, the overhead for each OpenGL call was the killer. Newer APIs like Metal and DirectX 12 (as well as general increases in CPU performance) should alleviate this problem and will probably obviate or at least minimize the need for such batching. But if you want to render lots of objects on older devices your batching and sorting methods will yield benefits.