How forcefully should a single-player engine reject invalid data?

In a single-player game, when trying to build an entity out of components specified in external scripts, what do you think is more desirable to happen when one of the components is ill-formed:

• Should the engine skip over that component and let the entity live in the game world with only the components that were well written?
• Or should it not add the entity to the world at all if but one of the components is ill-formed?

Note, I'm not talking about logging the errors — that should come without saying — just about what should happen to the entity.

• Are we talking about single-player or multi-player? – o0'. Sep 11 '11 at 18:46
• @Lohoris Single player. – Paul Manta Sep 11 '11 at 18:49
• As most answers have already alluded to, who is your audience? Are you making a moddable game or are you talking about for internal development purposes? – Tetrad Sep 12 '11 at 3:13
• @Tetrad I'd like the game to be as modder-friendly as possible. – Paul Manta Sep 12 '11 at 5:47

If you're talking about a moddable game, then you might want to follow one of your suggestions above. But if you are concerned about rolling over your own errors, I'd say don't do it. I have become an advocate of Fail-Fast. If this is an error that you have created and must resolve before releasing, you should make the error obvious. The article that is linked to at the bottom of the wiki page is a good read on the subject with why failing fast is good, and when it should and shouldn't be used.

• I think this is a great dichotomy, differentiating between user errors and developer errors. It makes sense for the exact same reason that compiler errors are generally pretty easy to fix but runtime/semantic errors are the devil. – jhocking Sep 11 '11 at 20:14
• If the engine includes a way to modify the entities once the game has started, you should at least give yourself a chance to fix the error before you fail. – Blecki Sep 12 '11 at 2:36

The distinction between user and developer is not always clear in game development. Standard programming techniques like "fail fast" are not always advantageous, especially as team sizes grow.

For example, perhaps your technical artist has screwed up the shader for the targeting outline - broke the fallback, let's say, so it's only loading on SM4 systems, and he didn't notice because he's got a top of the line system. This results in some animations failing to load. Those animations are referenced by a particular spell your combat designer has written. Finally, your level designer is trying to get the spawns in place and those spawns all happen to be able to cast that spell - but now she can't place any of them in the world because their spells aren't valid because the effects aren't valid because the shaders won't load because the designers always have the worst computers.

So your demo isn't ready by 2PM and your investors wonder why you can't even get a single enemy in the game and your project gets shut down.

Or you choose the option where you log the failure but keep trying, and the game plays fine except some enemies spell effects don't appear - but the investors don't know what those are supposed to look like anyway, so they don't notice.

For that reason, I'll almost always advocate the first option - spawn as much of the entity as you can. There are cases for fail-fast - like if the data should never be edited except by people capable of doing builds (i.e. programmers and technical producers) and is always checked 100% at load, or if you're absolutely sure the person responsible for the problem is the person using the editor - but those are not the usual cases, and require a lot of technical infrastructure per se, which you might not be ready to invest in.

• I would like to think that the scenario you propose can be prevented with good source control systems. In your scenario, the artist has "broken the build". Anyone else working with the broken shader should be able to roll the shader back to an earlier revision until the problem has been sorted out. – John McDonald Sep 12 '11 at 1:08
• @John While good source control is a must, and sometimes a rollback is necessary to restore the rest of a project to a working state until a failure is resolved locally, it's rarely useful as a prevention mechanism and shouldn't be relied on as such. – user1430 Sep 12 '11 at 3:12
• @John: On large projects, builds can take hours (or an entire day). So you actually need a two pronged approach - you need not just source control, but binary control, so a non-programmer can roll back to entire previous builds. But of course, that has its own risks and costs, because now you're developing new content against other out-of-date content, with executables that might have other bad bugs. And even finding the right build to roll back to might take 30+ minutes - if your designers all have to stop for half an hour and fiddle with builds, that's big money being wasted. – user744 Sep 12 '11 at 3:44
• @Joe Wreschnig: That makes sense. So I guess there must be a point at which fail fast is no longer an asset. Either in the types of people that use it, or on projects of a certain size. I suppose it's up to the people in the team to decide what works for them. – John McDonald Sep 12 '11 at 4:19
• I'm not sure fast hard fail, which is really what the question is about, is ever an asset. Even on a single person "team", maybe I don't want to deal with the shader code because I really have to get this level done. Certainly write good error handling and log all the errors, but I almost always have a better idea of what's important right now right now than I did when I wrote the error code six months ago. – user744 Sep 12 '11 at 4:51

The user should be able to preview the entity he is going to import, and know in advance if there are errors.

You should somehow decide what errors should be fatal, preventing it to be added to the game, and which can be dismissed as warnings.

Of course if for any reason the imported entity might be somehow able to irreversibly alter the savegame data, you'd better require for it to be flawless.

I would suggest that in development, it should be noisy about invalid data. i.e. log everything somewhere it will be read. However, if your engine can ignore this and continue it should do so. You can have logic like

void setValue(int value) {
if (value < MIN_VALUE) {
log(value + " is too low, using " + MIN_VALUE);
value = MIN_VALUE;
}
if (value > MAX_VALUE) {
log(value + " is too high, using " + MAX_VALUE);
value = MAX_VALUE;
}
}


Even in a multi-player game this is acceptable unless you are assuming that one player is trying to cheat the system.

After releasing the software, you are likely to want to turn off this logging by default on the assumption the players are not going to read these logs.

• Assuming the logs present no performance problem, you should keep logging on in release builds and send the reports back to the development team. (Appropriately informing players, of course.) – user744 Sep 12 '11 at 16:36
• You should be able to record more concise logs for release. In development you can tend to be more verbose. – Peter Lawrey Sep 12 '11 at 16:54