How to design a game's software such that it is easy to unit test?

Question

Is it practical to use a testing framework like JUnit in a game development situation? What sort of design considerations can you follow in order to make your game more testable? What parts of a game can/should be tested and what parts should/must be left to human testing?

For example, if the game loop is encapsulated in one function, it seems like it would be terribly hard to test. I like to refactor out an "update" function that takes a time delta and moves forward the game logic; this allows some interesting tricks like the ability to slow the game down by feeding it fake, slower time deltas.

Answer 1

One of the tenets of TDD is that you let TDD in some cases influence your design. You write a test for system, then write the code to make that test pass, keep dependencies as shallow as possible.

For me, there are only two things I don't test as part of unit testing:

First, I don't test visual elements and how things look. I test that and object will be in the right place after it updates, that a camera will cull an object outside it's bounds, that transforms (at least those that are done outside of shaders) are performed properly before being handed over to the graphics engine, but once it hits the graphics system I draw the line. I don't like trying to mock out things like DirectX.

Second, I don't really test the main game loop function. I test that every system will work when passed a reasonable delta, and that systems work together correctly when they need to. Then I just update each system with the correct delta in the game loop. I could actually have a test to show that each system got called with the correct delta, but in many cases I find that overkill (unless you're doing complex logic to get your delta, then it's no overkill).

Answer 2

Noel Llopis has covered unit testing to the degree that I believe you are looking for:

• http://gamesfromwithin.com/stepping-through-the-looking-glass-test-driven-game-development-part-1
• http://gamesfromwithin.com/stepping-through-the-looking-glass-test-driven-game-development-part-2
• http://gamesfromwithin.com/stepping-through-the-looking-glass-test-driven-game-development-part-3
• http://gamesfromwithin.com/backwards-is-forward-making-better-games-with-test-driven-development
• (he also has many other articles about TDD)

With respect to testing the entire game loop, there is another technique to prevent functional regressions in your code. The idea is to have your game play itself via a replay system (ie. first recording player inputs, and then replaying them over a different run). During the replay, you generate a snapshot of each object's state for every frame. This collection of state can then be called the "golden image". When refactoring your code, you run the replay again and compare each frame's state to the golden image state. If it differs, the test has failed.

While the benefits of this technique are obvious, there are a few things you need to be careful with:

• Your game needs to be deterministic, so you need to be careful with things like depending on your system clock, system/network events, random number generators that aren't deterministically seeded. etc.
• content changes after the golden image was generated can cause legitimate differences with your golden image. There is no way of getting around this - when changing your content you need to regenerate your golden image.

Answer 3

I agree with both of the Jeff's and jpaver's comments. I also wanted to add that adopting a component model for your architecture greatly increases it's testability. With a component model each component should be doing a single unit of work and should be testable in isolation (or with limited mock objects).

Likewise, the other parts of the game which rely on entities should only rely on a subset of the components to function. In practice this means you can usually mock out a few fake components to facilitate testing these areas or you can simply compose partial entities for testing purposes. e.g. you leave out the rendering and input components because those aren't needed to test physics.

Finally, I would ignore the advice around performance that you get from some people in the game community concerning interfaces. Write the code well with interfaces and if you run into performance problems down the road you can easily profile, identify, and refactor to solve the issue. I was not persuaded by Noel's concerns around the performance impact of interfaces or the complexity overhead that they added.

That said, don't go overboard trying to test every little thing independently. Like most things testing and design are about striking the right balance.

Answer 4

I thought I would add a second answer responding to the OP's comment that user could replace unit tests. I believe that is completely wrong as the purpose of unit tests is not to assure quality. If you want tests in place to ensure your program's quality you should likely investigate scenario tests or invest in great monitoring.

(With great monitoring and a product running as a service it is indeed possible to push some "testing" onto the customer but you need to have a sophisticated system which detects errors quickly and rolls back the changes responsible. Most likely this is too much for a small development team.)

Unit tests main benefit are that they force the developer to write better designed code. The act of testing challenges the developer to separate concerns and encapsulate data. It also encourages the developer to use interfaces and other abstractions so that he tests only one thing.