Game engine and data driven design

Question

I've heard about data driven design and have been researching about it for a while. So, I've read several articles to get the concepts.

One of the article is Data Driven Design written by Kyle Wilson. As he described, it seems to me that the application code (i.e. the code for controlling resources such as memory, network...) and the game logic code should be separated, and the game logic code should be driven by external data sources. At this point, I can imagine that the developer would write some sort of game editor which accepts external data about in-game objects (such as character information, weapon information, map information...). The scenario design will be scripted by custom language/tool written by programmer to let the game designer create interaction between in game objects. The game designer will either use an existing/custom scripting language to write script for the game, or drag and drop tool to create game world. Example of tool approach I can think of is World Editor, which usually packaged along with Bliizard's games.

However, another article is against the use of Data Driven Design, The Case Against Data Driven Design. The author suggests to not let the game design driven by data, because it would take more time to develop a game, since the game designer has the burden of programming. Instead, there will be a game programmer to program the game freely from the sketch design, and is verified by the game designer after the game programming is finished. He calls this is programmer driven. What I think of this method is similar to the way I used to do: The game logic is the application itself, as apposed to the above idea, the application is the game editor, and the actual game is designed based on the tool.

To me, the first method seems to be more reasonable, since game components can be reused for many projects. With the second method which opposes data driven design, the game code belongs to that game only. This is why I think Warcraft has so many game genres in it, such as the original Warcraft and various custom maps, and one of the most famous: DOTA which actually defines a new genre. For this reason, I heard people call the World Editor is the game engine. Is this true how a game engine should be?

So, after all of this, I just want to verify that is there any flaw in my understanding about these ideas (data driven, programmer drive, scripting etc...)?

Answer 1

Making your game (or any software product) data driven is almost always a benefit. The only real con is that you may spend slightly more time building the relevant systems up front; it will pay off over the rest of your career as a programmer though (even if you don't reuse those same systems for that entire time, you'll reuse the techniques you employed to build them).

The challenge, and where the disconnect in those two articles you linked comes in to play, is what you elect to put in data and who you elect to give access to that data. Fundamentally, data driven design and development just means that you put information in external storage, load that information at run time, and act on it. Your application code does what that external data tells it to, rather than you writing application code that directly does what you think the end result should be. It's not a complex idea.

You don't have to build complex "component driven architectures" (as is the fad these days). Putting constants for tweaking physics (gravitational force, restitution coefficients) in a text file is data driven. Scripts (in Lua or something else) are data driven. Describing your level data in XML. Anything like that.

You can drive just about any component of software with data, and you can pick and choose which ones you want to do so with. Developer time is expensive; programmer time especially so. If you can save you or other programmers time by putting behavior and data in external storage and not requiring the game to be recompiled for every little change, you should. You'll save money and get things done faster.

Furthermore, you run a huge risk in attempting to make "designers design" and having programmers "make those designs a reality," forcing a programmer to exist in the iteration loop for a designer's job: you run the risk of making the programmer feel like he is just a code monkey, making trivial little tweaks for the design constantly. This can be massively demoralizing for a large majority of programmers, who want to work on interesting technical challenges and not on being a designer's proxy.

For your specific questions:

Is this true how a game engine should be?

A "game engine" doesn't really have a fixed definition. Generally it's the unified collection of underlying technology used to make a game, usually supported by a bunch of related tools (and thus quite data driven). But it varies fairly widely from context to context.

So, after all of this, I just want to verify that is there any flaw in my understanding about these ideas (data driven, programmer drive, scripting etc...)?

You seem to be more or less on the right course, although you're perhaps over-complicating what data driven design and development is by conflating it with component-based systems.

Answer 2

As the author of the second post, I would like to clarify a few points.

1. As Josh Petrie suggested, structuring your code to use data instead of just hard coding everything is always a win. I wasn't suggesting otherwise. I was pointing out that pushing everything on the designer is not a good idea. The term "data driven design" means different things to different people, so I probably should have been more specific when I wrote the original article.
2. At every place I've worked, we create data structures that are tweakable in engine. To make a change, we don't have to recompile the game. We can change the number dynamically at runtime. The data structures are often stored in code, but depending on who is changing them, they can easily be loaded from a "data file".
3. Most development environments support some form of edit and continue or module reload for C/C++.
4. Most game development studios have gameplay programmers. Their jobs is often to work with the designer in creating a fun experience. Their main concern is not technical challenges but rather crafting fun from code. I have worked as a gameplay programmer for many years, and I find this more interesting than just trying to solve technical challenges. My responsibilities have varied, but I have found my work most fulfilling when I am in charge of the implementation and I work with the designers on crafting something cool. The problem with designers coding or scripting is that programmers often have to sort out the bugs, which is one of the least fun things you can do as a programmer.
5. What works best for a studio depends on the game. If you have a long time to make a game, and you want to give your game legs for the mod community and create something huge in scope, then making a game that is completely data driven makes sense. A lot of games don't have that goal. They have to churn out a new game in two years, and unless they have a hit franchise, it is probably a different type of game than their previous work
6. What a "designer" does can vary from studio to studio. I've heard of a development studio that hires gameplay programmers from other studios, calls them designers, and has them script the game behavior. This sidesteps the problem of having people who aren't trained in programming coding/scripting.
7. There should always be a delineation between game logic code and engine code. As well, you normally want to have some sort of visual editor for object placement. I've never worked at a studio where enemy locations are hard coded. They are placed in an editor. Let me propose an example of what I'm talking about. Let's say the designer thinks up an enemy. Does the designer than script the behavior of this new enemy type? That is what I consider data driven design (in terms of what Tim Moss wrote about it). In the way I am proposing, the programmer works with the designer, they make a fun enemy together perhaps with tweakable parameters, and then they are placed in the level.
8. Native code written by a programmer is going to execute faster at runtime than a script written by a programmer, which will execute faster than a script written by someone with less technical savvy. This performance may or may not be important depending on what type of game you are making and what you are doing, but it is something to consider.
9. You can share game code between games regardless of what method you choose. I'm not really sure what you are getting at with this point. Even if you aren't using a scripting language or visual tool to define some behaviors, you should be architecting your gameplay code into reusable components as much as you can. There will always be stuff that isn't applicable to your next game, but every place I've ever worked at, when we start the next game, we start with the codebase from the previous - even if it isn't a sequel. We then keep the stuff that makes sense and remove the game specific stuff.

In the end, there isn't a right or a wrong answer. It's a question of how you and your coworkers are comfortable working. When I wrote that blog a while back, there was a lot of talk about how all the work should be pushed to the designers, and I wanted to write about how plenty of successful game companies that I knew of found a different balance.

Also, as a side note, my blog entry is 5 years old. It seems like a lot more studios are moving towards scripting languages and whatnot these days but are creating mature tools for debugging them, which was my main complaint. When I had written this, I don't think Unreal Kismet existed, which I haven't used, but it seems like they are trying to simplify scripting and it apparently has a debugger. (No idea how well it works though)

For smaller scale games, I definitely don't think you want to try and roll in a scripting language or similar functionality into your tech, but if you have a huge team and a lot of time to devote to technology, it's possible to do this right and the time investment may make sense depending on the way your development team likes to work. Personally, I will probably cling to C++ for as long as possible because for me, it's the easiest and fastest since I often have to try and work around "features" like garbage collection.

Answer 3

You should look the BitSquid Tech Engine. It is build using DOD concepts. The blog of Niklas Frykholm is very interesting. There are many articles on how this engine is designed.

At the GDC 2011, Niklas has made a presentation about Data Driven Renderer.