Modern Compilers Inlining and Header and Implementation Practices in Game Development

I was taught that in general practice its best not to try to beat the compiler, at least until its proven to be stupid. So in general, and since I was told that its generally only used as a suggestion to the compiler, the inline tag has never been a priority for me to use because I was taught that the compiler is pretty good when it comes to understanding the trade offs your system will have to make when it comes to inlining vs. looped calls to a function pointer for example.

But then I just read that functions cannot be inlined unless they are defined in the header file because otherwise the compiler won't see it when it starts to work on other header and cpp files. I did some more research and I found that some people were saying that this is the old way things worked and that modern compilers perform "Whole Program Optimization" or "Link-Time Code Generation" which is responsible for the construction of the obj files that I'm used to seeing which can then be linked together and optimized further such as allowing the compiler to see functions in the obj file that were once "hidden" in the cpp file and then inline optimize them where appropriate. This sounds great but I'm wondering which source to trust and was wondering if I could get an additional opinion on the matter.

On the other hand I was reading up on header only implementations and how they tend to be used in libraries to increase compiler optimizations and was wondering if its was also a standard practice in game development or if it was typical to separate declaration and implementation into different files when developing a game engine.

• Why would games be different than non game software? – Vaillancourt Jul 9 '20 at 11:48
• @Vaillancourt harder real time constraints that sometimes make the programmer sacrifice certain luxuries such as readability for optimization – Ryoku Jul 9 '20 at 11:50
• Have you encountered a problem in your game project where your current, readable code is not sufficiently optimized for your needs? Since optimization is so sensitive to the specifics of each situation, we're likely to be able to give you better advice if you show that particular case, rather than asking about "standard practice". The kind of sacrifice you're describing is often the exception to the standard, in places where our performance budget forces us to do something we might normally not. – DMGregory Jul 9 '20 at 12:30
• This is an industry where we can't even agree on what a "game" is, or what job a "game designer" does. It wouldn't surprise me if you can find both header declaration conventions in common use at different studios. So asking "what do games usually do?" is generally not as useful a question as it might seem. What you really need to know is "what should my game do?" and do answer that, we'll ask you "what problem are you trying to solve?" - is the way you've been going about it so far impacting your runtime performance or iteration speed in some specific way we can help you solve? – DMGregory Jul 9 '20 at 13:42
• Every application with a tight processing time budget will have the same problem. So your question should belong to StackOverflow. However, you might be interested in this question, this question (see the second answer) and this question – wychmaster Jul 9 '20 at 14:06

[I] was wondering if its was also a standard practice in game development

(I'll cover only this because it's the only part that is on-topic here.)

The standard practice in game development is to ship a game that can be played in a decent way on the target hardware.

This means that developers have to find the right balance between time it takes to produce a game (including a decent compile time -- developer's time == money) and how smooth it plays on the hardware (rightly optimized).

Given this angle, there is not really a way to define a standard practice w.r.t. creating header-only libraries in the game development industry. And I suspect that applies to other industries too.

• That being said, if you want to mess around because you don't need to ship your game, you can write it really the way you like and see for yourself :) – Vaillancourt Jul 9 '20 at 14:49

If you're really working at sufficiently critical parts of the engine, I recommend noinline to help optimizers instead of forceinline. BTW, the compiler is stupid in its own way just as we are. It doesn't know in advance what is a common case execution path and rare case one unless you use PGO, and that's very tricky with an interactive game unless you write all sorts of tests that simulate what users commonly do. We're stupid about things like register allocation (and maybe instruction selection if we're really ignorant about computer architecture), but we certainly know better (or we should anyway) than the compiler about how users are going to use the software.

Tell the compiler, "This function isn't going to be called repeatedly in a loop. So never inline it." That can help a lot, and I recommend reaching for this over forceful inlining if people are tempted towards the latter. It helps even with inlining because say we have a function like this:

void some_func(...)
{
if (common_case)
f1();
else // rare case
f2();
}


The optimizer, absent PGO, has no idea whether f1() or f2() is a common case even though you know upfront. So if they're both reasonably small functions, the optimizer will tend to inline both. Then some_func may cease to be inlined, and that can cut into performance and show hotspots in profiling sessions that can disappear if you noinline the rare case. Mark f2() with noinline and then both some_func and f1 will likely be inlined. So I actually recommend making noinline your best friend if you are tuning at this level, not forceinline.

It's also more intuitive. There's probably some legitimacy to the adage that optimizers know better than we do about what to inline. They have a lot of information like precisely how many machine instructions will result and how many registers are allocated that is very difficult for a human to anticipate. But we certainly know better what to definitely not inline which they will tend to inline otherwise.

As for what compilers inline, linkers can inline things since the late 90s (maybe before but that's the first time I became really aware of it). Shoving everything into a header file as a way to encourage inlining seems silly to me. Besides, if these things are really critical, I don't see why a dev wouldn't at least profile these things with profiler in hand. It's very quick to tell what is and isn't inlined there. The benefits of header-only files are predominantly convenience as I see it (I'll be happy to change my mind if someone shows me a case where inlining every single thing in a header file actually boosted performance).