Return to Answer

The abstraction we get from C# class inheritance, interfaces, virtual function calls, references, etc. helps a lot with stuff in the middle and left of that chart. And for many games, that's all you need. But some games have features at the right side of the graph, that demand the kind of high data throughput where these indirections are too costly, and the limitations/complexity of data-oriented approaches are a worthwhile trade for the speed we can get.

A good tweet I just saw helps put this in perspective:

Juan Linietsky
@reduzio

This is how I see abstraction/optimization balance in sw design:

Code executed less often is the large majority of the codebase and can benefit from more abstraction. Code executed more often needs better algorithms, and critical code (small minority) needs to be data oriented.

• Good data locality: each cache line you pull in contains not just the component you want to act on now, but also the next component in the buffer, so it's already waiting for us when we want it. The linear access pattern allows the CPU to predict and pre-fetch the next cache line we'll want, letting us make full use of the available CPU.

  This also works if a single system needs to access/modify multiple component types - using a Structure of Arrays layout, we can race down multiple of these packed buffers in parallel.

  In contrast, our OOP example has strewn the GameObject and Component instances more or less randomly around the heap - taking whatever slot was available each time we called new Foo(). When we want to know what component to access next, we have to look up a pointer from the list, check that it's valid, then try to read the memory at that location. That memory is very likely not already in cache, so we incur a cache miss - burning hundreds of CPU cycles just waiting for the component data to be available - that's time when our CPU could have been doing useful work!

• Good code locality: because we have one system executing again and again on a big batch of data, the code we need to run is also continually in cache. We might even be able to write our update functions so they update blocks of components at the same time - or the processor might be able to rearrange our code on the fly to interleave multiple loop iterations (it's scary what these chips can do these days!)

  But for our naïve OOP example, we don't know which component class's Update() we want to call until we call it. That means chasing a pointer to the component instance, chasing a pointer to its VTable, and chasing a pointer to the function definition. Then for the next component we come to, we have to do it all again, since it's very likely a different component type than the previous one. Since we haven't used this new component's code lately, it's been evicted from the instruction cache and we need to burn hundreds of cycles again waiting to read it back in.

In practice, this approach can yield large performance wins. I routinely hear from Unity developers who have observed 10x speedups when migrating code from conventional MonoBehaviour scripts (which act similar to the naïve OOP example) into Unity's Job System (which acts similar to the DOD ECS example).

• Good data locality: each cache line you pull in contains not just the component you want to act on now, but also the next component in the buffer, so it's already waiting for us when we want it. The linear access pattern allows the CPU to predict and pre-fetch the next cache line we'll want, letting us make full use of the available CPU and minimizing waiting on the slow memory bus.

  This also works if a single system needs to access/modify multiple component types - using a Structure of Arrays layout, we can race down multiple of these packed buffers in parallel.

  In contrast, our OOP example has strewn the GameObject and Component instances more or less randomly around the heap - taking whatever slot was available each time we called new Foo(). When we want to know what component to access next, we have to look up a pointer from the list, check that it's valid, then try to read the memory at that location. That memory is very likely not already in cache, so we incur a cache miss - burning hundreds of CPU cycles just waiting for the component data to be fetched from main memory - that's time when our CPU could have been doing useful work!

• Good code locality: because we have one system executing again and again on a big batch of data, the code we need to run is also continually in cache. We might even be able to write our update functions so they update batches of components at the same time - or the processor might be able to rearrange our code on the fly to interleave multiple loop iterations, so the fast parts get done while we're still waiting on the slow parts (it's scary what these chips can do these days!)

  But for our naïve OOP example, we don't know which component class's Update() we want to call until we call it. That means chasing a pointer to the component instance, chasing a pointer to its VTable, and chasing a pointer to the function code to run it just once. Then for the next component we come to, we have to do it all again, since it's very likely a different component type than the previous one. Since we haven't used this new component's code lately, it's been evicted from the instruction cache and we need to burn hundreds of cycles again waiting to read it back in.

In practice, this data-oriented approach can yield large performance wins. I routinely hear from Unity developers who have observed 10x speedups when migrating code from conventional MonoBehaviour scripts (which act similar to the naïve OOP example) into Unity's Job System (which acts similar to the DOD ECS example).