Modern Compilers Inlining and Header and Implementation Practices in Game Development

Question

I was taught that in general practice its best not to try to beat the compiler, at least until its proven to be stupid. So in general, and since I was told that its generally only used as a suggestion to the compiler, the inline tag has never been a priority for me to use because I was taught that the compiler is pretty good when it comes to understanding the trade offs your system will have to make when it comes to inlining vs. looped calls to a function pointer for example.

But then I just read that functions cannot be inlined unless they are defined in the header file because otherwise the compiler won't see it when it starts to work on other header and cpp files. I did some more research and I found that some people were saying that this is the old way things worked and that modern compilers perform "Whole Program Optimization" or "Link-Time Code Generation" which is responsible for the construction of the obj files that I'm used to seeing which can then be linked together and optimized further such as allowing the compiler to see functions in the obj file that were once "hidden" in the cpp file and then inline optimize them where appropriate. This sounds great but I'm wondering which source to trust and was wondering if I could get an additional opinion on the matter.

On the other hand I was reading up on header only implementations and how they tend to be used in libraries to increase compiler optimizations and was wondering if its was also a standard practice in game development or if it was typical to separate declaration and implementation into different files when developing a game engine.

Answer 1

[I] was wondering if its was also a standard practice in game development

^{(I'll cover only this because it's the only part that is on-topic here.)}

The standard practice in game development is to ship a game that can be played in a decent way on the target hardware.

This means that developers have to find the right balance between time it takes to produce a game (including a decent compile time -- developer's time == money) and how smooth it plays on the hardware (rightly optimized).

Given this angle, there is not really a way to define a standard practice w.r.t. creating header-only libraries in the game development industry. And I suspect that applies to other industries too.

Answer 2

If you're really working at sufficiently critical parts of the engine, I recommend noinline to help optimizers instead of forceinline. BTW, the compiler is stupid in its own way just as we are. It doesn't know in advance what is a common case execution path and rare case one unless you use PGO, and that's very tricky with an interactive game unless you write all sorts of tests that simulate what users commonly do. We're stupid about things like register allocation (and maybe instruction selection if we're really ignorant about computer architecture), but we certainly know better (or we should anyway) than the compiler about how users are going to use the software.

Tell the compiler, "This function isn't going to be called repeatedly in a loop. So never inline it." That can help a lot, and I recommend reaching for this over forceful inlining if people are tempted towards the latter. It helps even with inlining because say we have a function like this:

void some_func(...)
{
    if (common_case)
        f1();
    else // rare case
        f2();
}

The optimizer, absent PGO, has no idea whether f1() or f2() is a common case even though you know upfront. So if they're both reasonably small functions, the optimizer will tend to inline both. Then some_func may cease to be inlined, and that can cut into performance and show hotspots in profiling sessions that can disappear if you noinline the rare case. Mark f2() with noinline and then both some_func and f1 will likely be inlined. So I actually recommend making noinline your best friend if you are tuning at this level, not forceinline.

It's also more intuitive. There's probably some legitimacy to the adage that optimizers know better than we do about what to inline. They have a lot of information like precisely how many machine instructions will result and how many registers are allocated that is very difficult for a human to anticipate. But we certainly know better what to definitely not inline which they will tend to inline otherwise.

As for what compilers inline, linkers can inline things since the late 90s (maybe before but that's the first time I became really aware of it). Shoving everything into a header file as a way to encourage inlining seems silly to me. Besides, if these things are really critical, I don't see why a dev wouldn't at least profile these things with profiler in hand. It's very quick to tell what is and isn't inlined there. The benefits of header-only files are predominantly convenience as I see it (I'll be happy to change my mind if someone shows me a case where inlining every single thing in a header file actually boosted performance).