When to carry on with a buggy game state versus terminate the process?

Question

Imagine a situation where a level/game designer has missed assigning a prefab in one of the game objects' inspector configurations. (I am coming from Unity, but what I basically mean a reference to some object prototype).

In this case we are getting a NullReferenceException which could crash the game.

If we know that this could happen, we can prevent this exception and print something useful to the console.

How should we choose between these two methods?

• throw an exception and crash the process
• or just print something useful to the console for the developer to know.

The problem is when player would be playing the game, in case of an exception they would be thrown out of the game and it would stop working showing them what has gone wrong and to send info to the developer, but in case of console log they would be able to continue playing the incomplete version of the game.

It may seem like this question requires an opinionated answer, but to me it seems like there is a best practice for these situations and there are vivid pros and cons to each one.

My thoughts:

If player continues to play an incomplete game this will result in an incomplete experience which may lead to bad reviews or something like that. But when game is terminated at some point - player may be even angrier that game is not working at all.

My own conclusion is that you would rather continue with the game, but in case of any error logs you would display player that something has gone wrong in the UI.

Unity specific:

Unity usually runs fine after unhandled exception has been thrown, so there could be no point to this.

But the problem is that I have a choice between:

Debug.LogError("Something useful for the designer.");
return new ADummyObjectThatWillPreventFutherExceptions();

or

throw new NullReferenceException("Something useful for the designer.");

The second one seems more appealing since it's really hard or impossible to predict most situations when this could happen.

Answer 1

There are a few considerations here:

1. Can you ensure it doesn't happen at runtime, by construction? (yes ➡ do that instead)

   This isn't always possible, but in the cases where it is I'd argue it's better than either a hard crash or a soft fail. Not only does an error that cannot happen no longer have power to harm your game experience, it also frees you from spending time or energy checking for it.

   In Unity, you can use the [RequireComponent(typeof(Dependency))] attribute to guarantee your objects are created with the components they're dependent on attached. You can also use the OnValidate() message to ensure an object is configured in a sensible way at design time, and fix it automatically or error before the game is even run.

   If you're working with C# 8.0, you may even be able to enable Nullable reference types, designating certain variables to never be allowed to be null, and warning in many cases where nulls may be handled unsafely, minimizing the surface area for null reference errors to arise at runtime.

   Unit testing covering all the edge and corner cases of your systems can be another great way to catch many potential errors of this type in isolation, before they become hard-to-reproduce bugs in your gameplay.

   See if you can incorporate this into your build/check-in pipeline. In the continuous integration system we use at the studio where I work, every new submission has to pass a battery of validation steps before it can be submitted, and before the submitted data is accepted into the next build. This catches common errors like invalid references in designer-made assets, so a whole class of data setup error is nipped in the bud.

2. Can this system do anything useful in this error state? (no ➡ hard fail)

   If you have a camera follow script, it doesn't make any sense without a target to follow. A character select screen can't let you continue with a null character selected.

   If the game isn't going to be playable past here anyway, you might as well throw a hard fail to make the exact site the error was detected as easy as possible to spot and fix.

   Something that arose in the comments: even if the camera example seems harmless/obvious enough to skip the hard fail, crashing here can be useful in particular if you have any automated testing in your game. Say I have a script that boots the game into each level/sector/mission beat to make sure it loads, and runs around randomly to see if it falls through an unloaded floor. Because this script is playing "blind," it's not going to notice if it manages to run right out of view because the camera follow script glitched, so it might report "loaded and played fine!" when a human player would find it anything but fine. Upgrading this to a crash makes it something even a very naive testing script can't miss, and it saves you from repeating the check in every test case if you trap it at its source.

3. Can this system do anything harmful in this error state? (yes ➡ hard fail)

   I'm currently wiring up some networking logic for a client-server game in a personal project. This involves a lot of coordinated state transitions and handshaking to be sure everyone stays in sync. If I manage to somehow find myself in a game state that I should not have been able to reach from where I was, this could be a Bad Thing™ - potentially causing cascades of additional errors that spill out into the server or impact other players, maybe even corrupting saved game progress.

   In this type of situation, a hard fail is a bulkhead to contain the error before it causes more damage. This type of error can also be a bit subtle and hard to track down if it runs away on you: the first observable symptom could be multiple game state transitions down the road from the first place you went off the rails, so Asserting at each transition that you've come from a valid state into another valid state can catch the culprit before you completely lose your map of the territory.

   Anything dealing with file access - and especially transactions or interaction with players' entitlements - is prone to this kind of consideration.

4. Is the error mostly cosmetic, at least for now? (yes ➡ soft fail / log)

   Small glitches will happen. We know this. Crashing/excepting on every undotted i or uncrossed t is going to slow down everyone's workflows and breed frustration. Especially if you're just prototyping a section that might not go in the released game anyway - you want to be able to be a bit loose so you can test the important stuff and not sweat the details.

   Timing matters here. What's just a cosmetic error early in development can become a compliance failure by the end, so expect to upgrade this class of bug later - maybe add a pre-processor directive or log level parameter that will let you escalate this from a warning to an error/crash when it comes time for polish.

   I'm currently working on a large open world game, and one of the most common errors I see is exceeding asset budgets for the number of props or vegetation objects in the current loaded chunks. This isn't really that much of a problem on our dev machines, which are built well above the game's minimum target spec. And if the framerate starts to chug a little bit, but folks can still do their work and test, then that's better than crashing and rendering a section of the map unplayable until the artists have time to fix it. (And also better than culling props past the budget limit - props that might be required for mission completion). So we just log warnings "Too many Asset type Xs loaded. The collection will expand in dev mode, but this will not work in release."

   Now, when we come closer to shipping and our game needs to hit our target framerate on the fixed hardware of a console, then we do a more stringent pass to prune whatever's over-budget, or outright fail validation of chunks that cross the limit.

   This phased approach lets us dial in the gameplay experience and look first, then get more strict about ironing out the technical details once that foundation is secured.

5. None of the above? Use your judgement.

   There's not a hard & fast rule for everything. At some point you will need to make a judgement call, or lean on a "house rule" policy or convention about how your team typically handles this type of scenario. Being consistent with how it's handled elsewhere in your project minimizes surprises and the risk of missing an important error/fix.

Answer 2

Let me get philosophy out of the way. If you think that the goal of game development is to deliver an experience to the players. Then that's it. Remove or hide anything that would break that experience.

Instead, think of giving a good experience to developers, designers, testers and players.

---

Now, I can be a bit more pragmatic. DMGregory answer is even more pragmatic.

I present to you the concept of environments:

• Development environment
• Testing environment
• Release environment

These might map to phases, or they might be custom builds.

On your development environment, you want to fail fast. The sooner the developer is aware of a mistake, the faster you can fix it, and the cheaper it will be. The cost of fixing bugs grows with time.

On the testing environment, the focus should be on getting the most information. If you can get stack traces, screen shots, a list of recent input, and so on, then do that. That would make it easier to track down those bug that did not happen often enough.

On the release environment, you still want to get as much information as possible. But you should not inconvenience the player. Furthermore, privacy and law comes into play when collecting information. That is something you should address in an EULA.

Thus, I'm saying that the way you deal with an exception, should depend on the environment.

---

You should also consider the type of exception.

We have exception from which you can't or it makes no sense to recover (such as a OutOfMemoryException).

Then there are exception that you should prevent instead of handle. These are exceptions for developers and should not reach Release (such as NullReferenceException). It could be that the developer missed them and they got into a testing environment. Code review, static analysts, and tests will help you find them.

On a test environment, you want to gather as much information of how they happened. So that the developer can figure out how to prevent them.

And finally there are exception that you should not prevent, but handle. These are exception about I/O (such as FileNotFoundException). For these, get a log, and try to recover and keep the game running. Note that even if you check if the file is there, the OS could preempt the thread after the check and when you use it. That gives a chance to another process to delete it.

Exceptions ere, of course, exceptions. Some systems will hide them, leading to a different exception down the road, one that should not be. It is exceptional. For example, the system loading a corrupted file should throw. Yet, they could have made it return null instead. That will lead to a hard to debug NullReferenceException, which could make its way to Release.

Oh, and null in Unity is a special case, because it isn't actually null.

This reduces the question to: how to handle an exception that should not reach Release, but did anyway? Well, I have one more concept for you…

---

Defensive programming and Offensive programming. Defensive programming is about checks and validations to prevent errors. While offensive programming is about letting errors happen and either crashing or recovering.

In Release. If you are dealing with external systems, be Offensive (e.g. do not try to prevent I/O exceptions, recover from them). If you are dealing with internal systems, be Defensive.

For example, a saved game is something you did not release with your game, and thus treat as external. Thus, when loading it, the guideline is to use offensive programming. No checks. On a corrupted save file, the system should throw. If it loads well enough to run, and if there is something missing because it failed to load, let it crash.

But a stage is something you did release with your game, and thus you may treat as internal. Thus, when loading it, the guideline is to be defensive. Check that it loads, if it didn't, have a fallback, or disable the content. For example, dialog fails to load, have some default dialog. Nobody should notce.

Note: You do not want the fallback content in the testing environment. Instead, you want testers to catch the problem giving you a chance to fix it.

Answer 3

You should always handle exceptions if you know what they are about, and crash if you don't.

Players overwhelmingly prefer a crash to a corrupted save.

If you want just one single rule that handles any and all unhandled exceptions, the only feasible one is to crash. Sure, it's far better if there is more graceful handling in case of minor/cosmetic bugs - but to do that you need to figure out which ones are minor/cosmetic.

You can handle minor/cosmetic bugs by writing to the log and or implementing a "temporary" workaround. But to decide if a bug is minor/cosmetic you have to first know what the bug is.

---

it's really hard or impossible to predict most situations when this could happen.

If there are lots of unhandled exceptions that are impossible to predict, there are major flaws in the software that must be addressed. If there are just a few unhandled exceptions, you find them with Unit Testing, then internal testing, then alpha and beta testing.

Answer 4

Pain is useful

Check this explanation of the 4 types of exceptions: https://ericlippert.com/2008/09/10/vexing-exceptions/
Out of these, it seems that you are talking about the second type; boneheaded exceptions.

are your own darn fault, you could have prevented them and therefore they are bugs in your code. You should not catch them; doing so is hiding a bug in your code. Rather, you should write your code so that the exception cannot possibly happen in the first place, and therefore does not need to be caught.

It sounds extreme, but exceptions are like pain. If you hold your hand into an open flame, you don't want to just be thinking "Hu, that feels funny. Anyway...".

Development vs. Release

Like you say yourself, "Unity usually runs fine after unhandled exception has been thrown", so during development, catching boneheaded exceptions doesn't give you a considerable advantage. Even worse, it does add more work to do. However, your text seems to be talking about a released game. Contrary to intuition, it's even worse there!

You set the example of a missing prefab. Let's say it's the graphics for your armor in a 2D platformer. Now that armor isn't being displayed, but you can still play. So we catch the exception, write a log message that the armor prefab is missing and accept it as a glitch.

Now the player proceeds and they update their armor. They proceed to play and it seems fine. Then they are done for the day and save it. The next day they try to load the save and it's corrupted.

So now we get a bug report about a corrupted save game. There's a certain, more or less specific sequence of actions that lead to this. Because of our attitude towards exceptions, a couple of other exceptions have been thrown in between - the one about the armor seems like the least likely candidate.
And now you may try and remotely debug this issue, via a third party that is probably a non-developer player, starting with a description of "Loading saved game doesn't work".

In all seriousness

The question is of course how "serious" you are about your game - are you even planning to support your game after release? If you're making something similar to Flappy Bird, or those 2000s flash games, the extra effort of fixing exceptions is probably not worth it.

Answer 5

The short answer from me is: You shouldn't ever carry on with a buggy game state. Anything you do carry on with needs to be considered "acceptable" at minimum. While @Theraot was on point with the discussion of environments, in my opinion it's a little bit more simple. Instead of what environment you're running, consider this:

Who is seeing the error?

• A Developer: Fail fast, and fail hard. Any unfixed bugs that make it past the developer will only become a bigger headache for people who know nothing about the code. I won't touch on runtime assertions here, but in my opinion it fails under the "fail-fast" mentality (Per @DMGregory: If the error can't happen, you never need to worry about it!).
• A Tester: Similar to developers, but provide more information than typically necessary, to save time in hunting down the details of a bug once reported. This would be akin to printing stack information, variable inputs, etc.
• A User: Again, fail fast. In this case, however, the end-user is very unlikely to be able to make use of the information that would be presented to a tester or developer. They need a new class of feedback.

In terms of responding to a user about crashing feedback, let's go off into a real world example, Minecraft's older error reporting system: Hopper.

On a game crash, a user would be presented with a plethora of information that was seemingly useless to anyone code-illiterate (read: many young children playing the game):

To alliviate this, Mojang added Hopper, which would receive a report from the client of the unique type of crash (e.g. you could hash the stack trace). The service can then have a landing page for those errors, which would provide more specific, user-level feedback about the crash, and how to fix it.

The main point I'm trying to drive home here is that lots of information can be good and helpful. However, in a production environment, sometimes more information can actually just mislead and confuse people, versus a simpler explanation from someone who understands the context of the issue. This also allows you to explain real-world crashes that aren't fixable by developers (e.g. not having graphics drivers installed), and provide remedies for those issues to your end-users.

You should not accept a buggy game state, as simply letting a bug continue onward in your game can leave as bad of an impression on users as your game crashing. But game crashes are much more obvious and likely to be fixed than silent bugs, motivating your developers to not push a buggy/crashing build to begin with.