How can I organize render and transformation data in a scalable fashion?

Question

I am writing for OpenGL 2.0 and in the future porting to OpenGL ES 2.0. I only use VBOs and shaders (no immediate mode, no vertex arrays).

I already have working solutions, they just... feel wrong. All my calls related to OpenGL are tucked away in a separate class called RenderManager (this includes creation of VBOs and IBOs, texture IDs and shader programs) and rendering. I called the creation of OpenGL objects logging in and out, so majority of this action is invoked from the ResourceManager object upon loading of data. However, a lot of geometry (level) is generated dynamically and therefore some RenderManager calls to create the VBOs/IBOs is scattered throughout the level class.

Now, I have a Renderable interface which returns the (single) transformation matrix and a RenderOp object when it is time for rendering. RenderOps is an object that has Vertex/Index Buffer pointers (has actual geom. data which in turn has VBO/IBO indices) and some parameters such as whether to use the index buffer and if the RenderOp needs blending and stuff.

There is also the RenderView object; it has the viewport parameters, projection matrix and view matrix (camera).

The game objects reside in the World class (single-dimension arrays). These game objects are added upon their creation, and are derived from Entity interface. Entity is able to return the number of Renderable-type objects within as well as the actual pointer to a Renderable at a given index.

The magic happens when the main loop traverses the World. First it collects all Light objects visible from within Frustum. Then it collects Entity object that's within Frustum (it gets added to std::vector inside RenderView if it is), and then it checks that same object to see if it is inside any of the visible Light's bounding volumes (and gets added into Light's std::vector).

Then the RenderView is passed onto the RenderManager class, which extracts appropriate Renderables from each Entity, applies Renderable's shader, then applies Renderable's textures, then stores the transforms inside itself and promptly shoves them down the programmable pipeline's throat along with attributes stored inside the RenderOperation (that gets extracted from Renderable as well).

Last note - I have NO way to render debug data, which is extremely annoying.

A lot of what I am doing feels wrong. My question is, how do you organize all your Renderable and Transformation data to facilitate your rendering needs? I am burning out thinking of semi-flexible ways of managing this data and I want to hear your ways. I am not a big fan of scene graphs, and consequently don't really want to implement one as I feel there are no real hierarchies inside the game, but I feel like I'm running myself into a corner.

Please throw me a bone here, how do you manage all this data? And if you suggest an event based rendering system, how would you design it?

Answer 1

I'm not a professional and my answer is not OpenGL-specific, so take things with a grain of salt, please.

That said, what I did in my own little rendering framework was this:

Everything that would qualify as a "GameObject" (in a strict OOP perception model) is called an entity. Entities are represented by just integer keys retrieved from an entity store. Entities can consist of any number of components. Those contain the real functionality and would be Mesh, Material, Transform, CollisionVolume, AudioPlayer, etc. When one wants to attach a component to an entity, they hand the entity key to a store for the wanted component. Behind the scenes, that store creates another component in a linear array of same-type components and remembers that whenever someone wants the retrieve a component with the same entity key, it's the one just created.

That way, if some components can be "updated" in bulk without knowing about anything else (Collision Volumes for instance) they are all linearly laid out in the component store's memory and if someone wants to retrieve just one of them, they can do so via the key.

So much for the general thing. Now regarding rendering, I'm still left in some kind of halfway immediate mode, where I tell the renderer the entities I want rendered via entity key. When I'm done declaring entities to be rendered, the renderer fetches all the components required for rendering, e.g. Mesh, Material, Transform (wich is Scale, Rotation, Transformation). Mesh- and Material-ID as retrieved from their component stores are aggregated into a sort-key, so that "entities to be drawn" are then grouped by similar meshes and materials. The Transforms have to sorted alongside so I can pack them up and provide them to instanced draw calls where many entities with the same mesh are submitted with just a single draw call and all necessary position data. Christer Ericson has a great post on this on his blog.

I'm just throwing this out, not knowing whether that last bit is a good idea at all (in OpenGL ES, anyway). I'm also aware that I glossed over a lot of details but it ought to be kept simple so you can evaluate the idea. Anyway, this whole thing should scale pretty well (provided the component stores and sorting mechanisms are implemented efficiently) and should also lend itself well to be data-driven as one just needs a mechanism that reads entity/component descriptions and then hits the stores to plug the components together.

One big question mark for me is how to handle interaction between components, but you wanted to know about rendering, so that should do for now :)

EDIT: Check out Niklas Frykholm's blog for a clever way to manage associating entity-keys to components.