Should game data be encoded in configuration files, given that it will almost always contain logic?

Question

Question inspired by the accepted answer to another question: Wondering if there is a more efficient way to store level data in my game?

The answer, by @Evorlor, says:

You should have the data for your level in an external source, such as an XML, JSON, YAML, CSV, or even a .txt file. Then you write your logic to read from the file, and generate the level procedurally.

This will let you play all of the levels with the same code, and you can make changes to levels, or add new levels, without touching the code. This also lets designers who aren't familiar with programming to create fun levels.

This also opens the door for user-generated content. You can create a level editor that auto-generates this file. A level editor will also make it easier to make fun levels, since they can be visually designed, instead of by writing text into a doc.

I have great doubts about this line of reasoning.

Years ago I tried (and failed) to make my own Pokemon-like game. Inspired by The Battle for Wesnoth and its WML (Wesnoth Markup Language) config files that could encode entire scenarios, units and abilities I thought I'd do the same. I'd encode monsters, abilities and types in configuration files; then my game would read these config files on startup. There'd be next to nothing hardcoded.

This proved to be a terrible idea. Especially abilities proved to be absolutely horrible to encode in configuration files. I started seriously doubting if I was really doing the right thing when I saw <if></if> and <not></not> tags in my config files. But how else was I supposed to implement abilities that place a debuff (that can do absolutely arbitrary things) provided another debuff is already in place? I've also run into problems with composition: Debuff A was working properly on its own, Debuff B was also working properly on its own, but when both debuffs were placed at the same time there were no longer working properly: it turned out that iterating the list of buffs and debuffs on each turn and applying each of them separately was not always working. So my code grew complex and unwieldy when, in my homebrew config files encoding a debuff I had to test for the presence of other debuffs, etc, etc.

Then I read The Daily WTF article on Soft Coding and I thought I found my error. In fact, my config files syntax grew to become my own programming language, but one that was deeply inferior to what I already had, namely C#. I ditched config files and instead moved all data to .cs files. It turned out only monsters remained trivially serializable to a data format; abilities, damage types and (de)buffs were so cluttered with logic that it was nigh impossible to serialize them (at least not without creating my own programming language based on XML).

The aforementioned question and answer deals with levels in an SMB-like game. OK, I yield that serializing level data could seem more affordable than, say, serializing enemy data. Still, I think I can foresee imminent difficulties. What if I wanted to put in my level, say, a catapult that can launch 'Mario' to some otherwise inaccessible parts of the map? Note that, inspired by Wario Land, I have put a Rusty Spring as a treasure in this very level; but also I have put a Titanium Powered Adamantium Spring as a treasure in another level. Depending on which spring the hero brings to the catapult, it operates differently. With no spring, it is not functional at all; with a Rusty Spring it launches the hero to a secret area; with Titanium Powered Adamantium Spring it launches the hero to another, even more dangerous bonus area that is filled with even more treasure; with both springs, the player may chose the area they wish to be teleported to.

Not to mention typical various switches, booby traps activating / deactivacting on conditions, NPCs saying different things depending on if you've beaten a boss (or another arbitrarily complex conditions), etc, etc. Level data can easily become cluttered with logic!

If, from the very beginning, I put my level data in config files, I will start banging my head against the wall whenever I decide to implement any of the aforementioned features. If, however, I never touch the 'config files' route and instead always keep level data hardcoded (even when it still seems that this data is trivially serializable) I will not run into problems when I have an idea to implement any of the aforementioned ideas.

Which is probably why level editors in various games typically allow us to define variables and conditions by clicking through their GUI. Essentially, a level editor doubles down as a... visual programming language.

It seems to me that the cost of developing such tools is enormous and should be very carefully weighted against any possible gains from doing so. The situation where making a program (any program, not just a game) more configurable leads to defining and implementing my own programming language (whether by overly complex config files, overly complex config menus or both) should be avoided at all costs.

In particular, if someone just tries making their first game, then this is one of the first features that should be cut (under the general rule to scope smaller).

But even if this is indeed necessary, then game logic should still not be put in config files, but perhaps in Lua code (this is also how, if I'm not mistaken, Factorio can be modded; and also Wesnoth started moving logic in user generated contents from ther Wesnoth Markup Language to Lua).

Do I fail to see something? Why is it recommended that game devs, by default, put their game data in config files? Is it not the case that, over time, these config files will have to store logic rather than just data, making them unwieldy?

Answer 1

A game or an engine

If you were making a game, then you write the code for each case in your game. Then the configuration only needs to specify which of the possibilities it uses. Which seems to be what you ended up doing.

On the other hand, if you were making an engine or a game to be modded, then it makes sense to provide some scripting language (e.g. Lua, as you point out). The configuration system can remain simple, by having it point to a script file if necessary. So, no, configuration does not have to become a programming language, nor become unwieldy over time.

To give you a world of nuance, I believe some particular cases will look like programming anyway. See dialogue at the end of the answer.

Even if you are providing an script language, whatever you find is implemented often is probably better to be implemented in the game proper, in such way that the script can call it. Thus, the complexity of the script files would also be bounded.

However, advocating for configuration is not advocating for scripts. There is nothing wrong with having all the logic hard-coded in the game.

---

What do you gain with configuration

It might be worth remembering that the context of the question that inspired this one was a web game. It would download files from a server. In that context there is no clear distinction between hard-coded and loaded in runtime.

In fact, There is nothing wrong with hard-coding the scenario description.

Furthermore, "configuration" does not have to be loaded from storage in runtime. In fact what we have been calling "configuration" can be arrays hard-coded the source code. What we really want is the ease of reading it and modifying it (i.e. easy to maintain).

If you have a hard-coded string or array of characters where each character represents a tile, and you arrange it in a way that resembles the shape of the scenario, you might be able to visualize it directly, which makes it easier to modify it too.

Take the example by Evorlor, it could easily be hard-coded:

var scenario = "" +
"                                F" +
"          ?*?    C              F" +
"                                F" +
"     M           G              F" +
"======================    =======";

And even hard-coded it remains relatively easy to visualize and modify.

And it still requires the code that iterates over it and instantiates the real thing.

While this is not the full advice, doing this already gives you a game that is easier to maintain. There configuration is not the goal, it is a means.

---

Of course, you can have some file that you can open in some external editor (which might be visual or not). And it still requires the code that iterates over it and instantiates the real thing.

Having a separate file, while still being part of the source code gives you one more perk: you have separate diffs in version control for changes to the scenario and changes to the logic.

Furthermore, as pointed by Evorlor, it make it easier for designers who are not familiar with coding:

This will let you play all of the levels with the same code, and you can make changes to levels, or add new levels, without touching the code. This also lets designers who aren't familiar with programming to create fun levels.

Thus, a third perk is to allow designers to create and modify content without messing with the source code. Again, configuration is not the goal, it is a means.

I disagree with Evorlor that a configuration file "opens the door" for content generation, although I agree it makes it massively easier. For example, the approach shown in the question that inspired this one would have been easy to modify to include randomness. However, a configuration file creates a nice boundary between the logic that instantiates the level and the logic that generates it, which will result in much simpler code.

---

About tools

Creating your own tools is an investment. But that does not mean creating a tool is always a good investment.

It makes sense to pick existing tools first. Consider if there is some format you can load for which there are existing editors. There are editors for tiles maps, and interactive fiction, and so on, that use format that you might be able to use in your code. In fact, there might be plugin for whatever engine you are using already.

An advice similar to that we often give about optimization of code applies here: find the bottlenecks of your development process and focus on those. For example, if you find that a lot of time goes into modifying scenarios, then perhaps developing tool for that is worth it.

---

Making a powerful scenario descriptor

First of all, I remind you that these are means to the goals. The goals in this case include supporting the different situations you describe where complexity can become hard to manage. But what I describe below is not the only means to those goals.

This more of a "hey, this works for me and other people like me" and not a "this is the law of how to do things". OK? OK.

---

Scenario and entities

To begin with you have a description of the scenario. Presumably this description includes the starting position of enemies. The behavior of enemies - hopefully - is in the core game, and the description of the scenario only specifies which one it uses.

Well, scratch the word "enemy". The description includes the position of entities. And these entities can have arbitrary logic. Perhaps they have an associated class, from which an instance is created to handle each one - or something more fancy, for example they might be entities of an ECS - or, sure, they execute a script loaded at runtime.

Thus, the catapult you describe is an entity. The code for it is not part of the description of the scenario. And it does not have to be loaded from storage in runtime.

If anything the behavior could be configured. That is, the description of the scenario can include values that are passed to the code in runtime. Furthermore the entity code might be able to query the scenario in runtime. So you don't implement bird that flies at height 3 and bird that flies at height 8, you use a variable. Similarly, you don't need warp that goes to level 2 and warp that goes to level 5, you use a variable, or multiple.

The catapult in particular would need a list of keys and values that map items to destinations.

---

Game state

So you need switches that activate things, and those things are other entities.

In the simplest case you need a way to refer to entities in the description of the scenario, so you can write that such switch points to such door or whatever.

However, as it turns out, it is more flexible to have an intermediary. You will have a global game state (e.g. a blackboard) where the switch can write, and the door can read. And it might also send notification when data changes, so that different entities can subscribe to react to the changes. With that you can implement a switch that open multiple doors, or a door that checks multiple switches... Plus, this global game state can be serialized to save the game! And load it continue!

Of course the global game state can become a mess. The minimum implementation would resemble a key-value storage. However you will find it useful to have hierarchical keys. So you can give a sub-key to specific entities, or players (plural in case of multiplayer) where they store their own state, while keeping it all collected in a place from where you can easily save it and load it.

---

Dialogue

How to write interactive fiction is a thing of its own. Most games will have the dialogue hard-code because it is part of fixed story beats that the game always follow. But that is not every game. In fact, configurable dialogue is a powerful tool for designers making quests.

So here is my approach: You have storylets. Each storylet has:

• Has some preconditions.
• Have some effect.
• Text, and whatever else you need to show the dialogue (portraits, etc).
• Plus I find it useful to have a list of functions it can call asynchronously (e.g. move the camera, fade, change the music, and so on).

The preconditions can check the global game state (e.g. such boss has been defeated) the state of the entity (e.g. what have you talked already) and the player (e.g. if they have some item in inventory). This would all be part of the global game state if you follow my advice of giving sub-keys to entities and the player, except you will use some shorthand to specify "hey use the sub-key of this entity or this player"

Similarly the effect can write to global game state, the entity, or the player. The changes would be setting the value of keys to new values, and the change can be relative (i.e. increment or decrement).

Plus, always have as part of the state of the entity if it has already said a given dialogue (so that you can always have it as precondition for another dialogue, and that is how you chain them).

Then the entity has a pool of these storylets. And it has some strategy to pick them, for example, it might go in order, or at random, or you might implement a priority system, or something else. You might have logic for the different strategies and write in the description of the scenario which one the entity uses.

---

Thus, yes, I believe dialogue will have expressions (e.g. comparing values, assignments, or calling a function - from a predefined list, of course). Although the programming paradigm is not structured (it is not based on if-then and similar statements)... In fact, storylets have no branching.

With that said, modern interactive fiction tools use custom programing languages. Yet, I believe it is still better to have anything that requires complex logic taken out of it... So that it does not grow too complex, writers don't need to be programmers, and changes to the dialogue are separate from the rest of the code.

Answer 2

Moving game data into config files has several advantages:

• You can edit these files without having to recompile the game. Which can, depending on your technology stack and the scope of your game, save you a lot of annoying waiting time in your testing loop. With a bit of extra work, you can even make data files hot-reloadable, so you can experiment with them without even having to restart the game.
• You can edit these files without having to know the programming language and the development tools. This can be extremely useful if you aren't working alone but collaborating with people who are not programmers.
• You are not bound to the syntax of your programming language. Not every programming language is equally good at encoding data literals in the shape you need and in a way that is easily readable and editable. So moving that data to a configuration file can make your life a lot easier.
• You have the option to create editing tools that allow you to create content a lot faster (whether or not you should is a question of how much time it would take vs. how much time it would save. These are highly dependent on the project).

However, I agree with the TDWTF article on "Soft-Coding" logic to a level where the configuration language basically becomes so convoluted that all the above advantages are lost. When everything is soft-coded, then nothing is soft-coded. It's hard-coded in a proprietary programming language that is often worse than the language that was used to build it.

My general rule of thumb: Logic belongs into code. Data belongs into files.

But very complex games made by large teams often do move some logic to script files. That way they create a layer between the "arcane", highly technical code of the core game and the "dumb" configuration files that are just data. A scripting language you can teach to level- and game designers that allows them to do trivial logic without asking a programmer for help, but which isn't powerful enough to seriously break things. A common example are dialog trees in RPGs. If a writer wants the NPC to say one thing if the player made decision A and another thing if the player made decision B, then they shouldn't have to bother a programmer with that. Especially when there are about a hundred situations like that in the game.

Where to draw that line between what to program and what to script is a judgment call. There is really no hard rule to follow here, except the obvious one: Do what you think is less work.

To take the example of the one-off spring catapult thing from the question: I would turn this thing into an entity that can be placed in the level files. Even when it's just supposed to exist once in the game. That way the level designers can easily experiment with where to place it exactly. It also makes sure that when you look at the level file, you get the full picture of what is and isn't in that level. You can see that this empty plateau in the middle of the level isn't just wasted space, because that catapult is there. However, the actual logic behind it, might work better if it is hard-coded in a class.

Answer 3

Only externalize if you solve a problem with it

Configuration files and similar external resources are an option which should be used if the benefits outweigh the costs. Generally speaking I would always start with the simplest solution which solves my problem. Refactoring a simple solution to a more complex one is usually not very expensive. On the other hand using an unnecessary complex solution for a simple problem is a waste of time.

To decide which is the easier approach consider Pro/Con of data in code:

• PRO: You are already proficient with code, have a nice editor, it is checked/linted and bundled automatically
• PRO: Needs no additional code to load, parse or handle errors or interact with the file system
• PRO: Easily extensible and customizable - you can add individual code at any point for any level
• PRO: If your language/debugger supports hot code replacement, you might even see your changes in real-time without restarting the game.
• CON: If hot code replacement doesn't work, you might have to rebuild your game to see changes, which might take longer (but is mitigated by incremental builds).
• CON: If collaborating with designers/community editing a simple file for data can not as easily break things as editing code. And if the file is read on game startup, collaborators can edit and test levels/enemies/… without any IDE installed/without building the game.
• CON: If designing a lot or big levels a good specialized editor might save you a lot of time. By using a standard-format you can use standard tools like level-editors

Weigh your options

Depending on the project and the type of game/data you should decide each time, if using an external file solves a problem for you. If it does then use it - if it doesn't then just keep the data in your code.