Data-oriented design and Component Sytem cross-referencing

Question

Let's assume we're talking about a game (engine) which is written in C++. This is more like a design question but I can't find any suitable description.

Component System

The Component System says that an Entity in the world is defined by its components. This system is flexible, modular and really powerful. I've used Unity (a lot) and I like its concept. This also solves the traditional class-hierarchy problem.

Example components:

• StaticTransform: Position, Rotation, Scale - a StaticTransform can be a child (and of course a parent) of another StaticTransform (this describes the Entity-hierarchy in the scene)
• Rigidbody/Transform: Position, Rotation, Force, etc. - I could use the StaticTransform data, but I'd like to separate the concepts. That's why I have Position and Rotation (but not Scale) data. A Rigidbody can not be a child of another object but can be a parent of StaticTransform.
• MeshRenderer: LocalBounds, WorldBounds, Mesh, Material - this component is used to generate draw calls to render the mesh with a specific material.
• AudioSource: AudioData - A simple audio source which can be world-space (using one of the transforms) or global (in this case the transform component is irrelevant)
• AudioListener: Only one AudioListener is allowed. Most of the time it's added to the player's Entity. It uses the transform's data.
• Camera: FoV, Clipping planes, etc. - Of course a Camera can be used to render the meshes from that point of view. Multiple cameras are allowed.

Data-oriented design (DoD)

The key concept of the DoD is to focus on data and data-access instead of code. There are some additional "rules" like: "When there is one, there are many". Most of the time this means that the data itself is stored in a cache-friendly way: contiguous memory with linear access (aka. iterating the array one-by-one). In a "Component System world" the component itself does not exist but the corresponding system stores and manages the data.

In a system like this, raw or managed pointers are forgotten, instead ids and/or handles (special id with additional info) are used. Most of the time a special entityId -> componentId map is used to match the entity to the component. This way the component data can be packed (moved in the array) so the iretation is fast (cache-friendly).

In theory this work really well. Blog posts and other discussions show examples how to do it with a single "component". However I cannot find any good solution to the cross-referencing problem.

The cross-reference problem

Using the example components described above: The MeshRenderer depends on the (both static and dynamic) Transform component because the World matrix is requried to render/cull the mesh. The AudioSource, Camera and AudioListener also depend on the Transform component.

The entities can be arranged into a hierarchy using the Transform's parent-child relation. When the user updates the transform of a root object, every child Transform must be updated as well. This means we are jumping back-and-forth in the memory.

Rendering all meshes all the time seems to be a bad idea. That's why people implement spatial trees which can be used to cull meshes and other renderable objects efficiently. Again, this kills the cache-friendly part.

Question

Should I even use the DoD concept in a complex system like this? The hierarchy, the spatial trees and the cross-referencing kill the linear access (so the cache/prefetcher-friendlyness) advantage. It seems using a "stable vector" (adding/removing elements and resizing the container do not invalidate iterators) is not worse than packed vectors.

Is there any good solution to the problems described above? I like the key concept of the DoD but I'm failing to apply it.

Answer 1

I would not necessarily mimic in the component system the necessary data structures your logic systems require to function. There is an advantage to keeping the component system straight forward and simple and using a layer of indirection here.

As an example, when the level is loaded, an entity is created that contains a Transform and a Mesh component.

A RenderComponentSystem is responsible for detecting this entity has been created, it examines the components of the entity and detects this Mesh component. This system is responsible for interacting with the RenderSystem so that the mesh is loaded and a specific RenderSystem data structure is created, I call a Renderable.

This RenderComponentSystem also detects that the entity has a Transform, and based on whether its a parent Transform or a child Transform, it interacts with the RenderSystem's scene graph to create a scene node and sets the appropriate position, orientation, and scale.

Finally, the RenderComponentSystem attaches the Renderable to the scene node.

How the scene graph is traversed or culled is simply a RenderSystem detail. From a component system perspective, it doesn't care nor is it concerned. The point is that the component system is meant to provide a simple abstraction for taking a data-driven component when composing entities and influence specific system behavior.

As changes are made to the Transform or the Mesh, the RenderComponentSystem simply iterates the components, clones/updates the attributes on the associated scene node or Renderable. This could be done by using a simple map-lookup or some associative pointer between the component and the render system's data structures.