26
\$\begingroup\$

Is it practical to use a testing framework like JUnit in a game development situation? What sort of design considerations can you follow in order to make your game more testable? What parts of a game can/should be tested and what parts should/must be left to human testing?

For example, if the game loop is encapsulated in one function, it seems like it would be terribly hard to test. I like to refactor out an "update" function that takes a time delta and moves forward the game logic; this allows some interesting tricks like the ability to slow the game down by feeding it fake, slower time deltas.

\$\endgroup\$
4
  • 6
    \$\begingroup\$ Why waste man-hours writing unit tests when you have a virtually endless army of slave labor to do playtesting for you? I kid, I kid... ;) \$\endgroup\$ Aug 18, 2010 at 21:58
  • 1
    \$\begingroup\$ Actually you need not kid, that's a really good point! I didn't think about that, but games are the easiest type of software to get other people to test. I guess that somewhat offsets the difficulty of automated game testing (unit or otherwise). \$\endgroup\$
    – Ricket
    Aug 18, 2010 at 22:01
  • 3
    \$\begingroup\$ Unit testing isn't necessarily about making sure your application works correctly as a whole (i.e. functional testing). It's more about making sure future changes don't break existing functionality. Sure, a user can see when the spaceship is upside down, but when the pathing algorithm is off by .001, the effects may not be apparent until the game is played for 200 hours. That's the kind of things unit tests can catch before the code ever goes out the door. \$\endgroup\$ Aug 21, 2010 at 5:46
  • 1
    \$\begingroup\$ Games are the easiest software to get users to play, but playing != testing. Getting good bug reports is quite hard. Beyond that, tracking down where in the code a bug occurs, and verifying that new changes don't break existing code both benefit dramatically from automated tests. \$\endgroup\$
    – munificent
    Feb 4, 2011 at 2:51

4 Answers 4

26
\$\begingroup\$

One of the tenets of TDD is that you let TDD in some cases influence your design. You write a test for system, then write the code to make that test pass, keep dependencies as shallow as possible.

For me, there are only two things I don't test as part of unit testing:

First, I don't test visual elements and how things look. I test that and object will be in the right place after it updates, that a camera will cull an object outside it's bounds, that transforms (at least those that are done outside of shaders) are performed properly before being handed over to the graphics engine, but once it hits the graphics system I draw the line. I don't like trying to mock out things like DirectX.

Second, I don't really test the main game loop function. I test that every system will work when passed a reasonable delta, and that systems work together correctly when they need to. Then I just update each system with the correct delta in the game loop. I could actually have a test to show that each system got called with the correct delta, but in many cases I find that overkill (unless you're doing complex logic to get your delta, then it's no overkill).

\$\endgroup\$
10
  • 6
    \$\begingroup\$ The first paragraph is key. The fact is that TDD requires you to design your code better than you would have without it, simply to allow the testing. Of course many think that they are experts at design, but of course those most impressed with their own skills are usually the ones with the most to learn... \$\endgroup\$ Aug 19, 2010 at 0:18
  • 2
    \$\begingroup\$ I wouldn't agree that the code is necessarily designed better as a result. I've seen plenty of abominations where a previously clean interface had to be smashed open and encapsulation broken in order to inject testable dependencies. \$\endgroup\$
    – Kylotan
    Aug 20, 2010 at 11:03
  • 4
    \$\begingroup\$ I find that happens more in instances where tests are written for pre-existing classes rather than the other way around. \$\endgroup\$
    – Jeff
    Aug 21, 2010 at 4:26
  • \$\begingroup\$ @Kylotan that's probably more a symptom of improper design/poor test design. Refactor and make use of mocks to make clean tests, don't hack your code into a worse state; that's the opposite of what's supposed to happen. \$\endgroup\$
    – timoxley
    Sep 11, 2010 at 20:10
  • 2
    \$\begingroup\$ Encapsulation is a good thing, but the more important is to understand WHY it is a good thing. If you just use it to hide away dependencies and a big ugly interface it pretty much shout out load that your design under the hood probably isn't very nice. Encapsulation is primarily not about hiding ugly code and dependencies, it is primarily about making your code easier to understand and to let the consumer get less paths to follow when he trying to reasoning about the code. \$\endgroup\$ Jan 3, 2013 at 19:18
20
\$\begingroup\$

Noel Llopis has covered unit testing to the degree that I believe you are looking for:

With respect to testing the entire game loop, there is another technique to prevent functional regressions in your code. The idea is to have your game play itself via a replay system (ie. first recording player inputs, and then replaying them over a different run). During the replay, you generate a snapshot of each object's state for every frame. This collection of state can then be called the "golden image". When refactoring your code, you run the replay again and compare each frame's state to the golden image state. If it differs, the test has failed.

While the benefits of this technique are obvious, there are a few things you need to be careful with:

  • Your game needs to be deterministic, so you need to be careful with things like depending on your system clock, system/network events, random number generators that aren't deterministically seeded. etc.
  • content changes after the golden image was generated can cause legitimate differences with your golden image. There is no way of getting around this - when changing your content you need to regenerate your golden image.
\$\endgroup\$
1
  • \$\begingroup\$ +1 It's also worth noting that the value of this approach is directly related to the length of the "golden image": If it's not long enough, you can easily miss bugs; if it's too long, you're going to have a lot of waiting to do. It's important to try and get the game to run perhaps an order of magnitude faster in this mode, than you would under normal runtime circumstances, to shave time off. Skipping rendering can help! \$\endgroup\$
    – Engineer
    Oct 20, 2015 at 7:40
9
\$\begingroup\$

I agree with both of the Jeff's and jpaver's comments. I also wanted to add that adopting a component model for your architecture greatly increases it's testability. With a component model each component should be doing a single unit of work and should be testable in isolation (or with limited mock objects).

Likewise, the other parts of the game which rely on entities should only rely on a subset of the components to function. In practice this means you can usually mock out a few fake components to facilitate testing these areas or you can simply compose partial entities for testing purposes. e.g. you leave out the rendering and input components because those aren't needed to test physics.

Finally, I would ignore the advice around performance that you get from some people in the game community concerning interfaces. Write the code well with interfaces and if you run into performance problems down the road you can easily profile, identify, and refactor to solve the issue. I was not persuaded by Noel's concerns around the performance impact of interfaces or the complexity overhead that they added.

That said, don't go overboard trying to test every little thing independently. Like most things testing and design are about striking the right balance.

\$\endgroup\$
2
  • \$\begingroup\$ Please note, though you don't seem to be new to SX, that answers are not ordered and saying something like "both of the above comments" is bound to quickly deprecate, as it currently is now that you are one vote ahead of one of the two other answers at this time. :) \$\endgroup\$
    – Ricket
    Aug 18, 2010 at 22:03
  • \$\begingroup\$ Thanks I didn't think about that while commenting. I've since updated the comment to pick out the individual comments. \$\endgroup\$ Aug 20, 2010 at 0:54
3
\$\begingroup\$

I thought I would add a second answer responding to the OP's comment that user could replace unit tests. I believe that is completely wrong as the purpose of unit tests is not to assure quality. If you want tests in place to ensure your program's quality you should likely investigate scenario tests or invest in great monitoring.

(With great monitoring and a product running as a service it is indeed possible to push some "testing" onto the customer but you need to have a sophisticated system which detects errors quickly and rolls back the changes responsible. Most likely this is too much for a small development team.)

Unit tests main benefit are that they force the developer to write better designed code. The act of testing challenges the developer to separate concerns and encapsulate data. It also encourages the developer to use interfaces and other abstractions so that he tests only one thing.

\$\endgroup\$
2
  • \$\begingroup\$ Agree, except that unit tests don't force developers to write good code. There's another option: really badly written tests. You need commitment to the idea of unit tests from your coders to stop them getting jaded. \$\endgroup\$
    – tenpn
    Aug 20, 2010 at 7:46
  • 2
    \$\begingroup\$ Sure but that's no reason to throw the baby out with the bath water. Use tools responsibly but first use the tools. \$\endgroup\$ Aug 20, 2010 at 21:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .