Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

We have several quite heavy game clients completely written in AS3 (some also use Flex for UI). Some clients are built with PureMVC, others are a pile of spaghetti code that's grown over the years. We also have several libraries used by our game clients that are written in AS3/Flex.

I'm tasked to evaluate the benefits/risk/cost of applying unit-testing to our clients and libraries and decided to give FlexUnit a closer look. I chose FlexUnit for a first attempt over fluint since FlexUnit is definitively able to test pure AS3 projects while I'm not sure about that for fluint.

Now I'm wondering what some good starting points and best practices are to get unit-testing into previously untested applications.

So my questions for unit-testing game clients and libraries are:

  • How to get started? What are "low-hanging" (= easily testable) fruits to test with unit-testing?
  • What are commonly tested aspects of game clients?
  • Can you apply unit-testing to the BBOY design pattern or do you need to (extensively) refactor the game clients?
  • Is it useful (and does it make sense) to test asset loading? If so, how to do it (comparing a binary diff, or comparing hash checksums, or ...)?
  • How to test server communication and asynchronous commands?
  • How to integrate the whole thing with Hudson/Jenkins CI server and Maven? Are there any reporting tools that can generate overviews of the run tests and report which test failed on which function/class?

I'm thankful for all pointers about the issue (direct answers, blog posts, forum discussions, how-tos, etc. etc. etc.) since I have little previous practical experience with unit-testing.

share|improve this question

2 Answers 2

up vote 5 down vote accepted

For me the easiest way to start testing a code base which doesn't have any tests, is to test bugs. When a you discover a bug, the first thing to do is write a test to reproduce it, then fix the bug to make the test pass.

Other low-hangers are classes which contain logic or data. If you're using the MVC pattern, test your domain models first, and then your controllers. Its often easiest to do this while re-factoring, because you will probably need to make changes to the classes to make them testable (using Dependency Injection and Single Responsibility principles).

For code which is not using patterns, see where re-factoring to a pattern might help, then approach it in a test-driven manner. Write tests for the structure you would like to have, then port your code into that structure to make the tests pass. At the very least, create a façade to act as an organised API over the spaghetti code. Write tests to develop the façade, then after it's working, write integration tests to get it working with your current code. You can then re-factor or re-write the spaghetti code without impacting the rest of the system.

Testing client-server-interaction is also a big pay-off, as it is usually the most fragile code because it interacts asynchronously with a foreign data set. Start at very coarse granularity. So first test that the client-server operations happen in the expected order. Then write tests to ensure that the data is sent and received in the correct format, then test how it breaks when used in unexpected ways. Write tests to call operations out of order, pass invalid data, etc. In FlexUnit you can use [Test(expects="Error")] when you are testing conditions that are expected to throw exceptions. You can also use [Test(async)] in conjunction with the Async methods to test asynchronous operations.

Testing third-party code can also pay off for two reasons: firstly it provides documentation by example and use cases for the library, and also indicates compatibility with other versions of the library.

The trick with writing ad-hoc tests (as with all re-factoring) is to do it in small steps, one line at a time until you have covered a method, then a method at a time until you have covered a class, then a class at a time until you have covered the entire library.

Another tip to keep in mind, is to test behaviour, or rather test what the code does, not how it does it. So if you have collision detection code, test that it returns a positive result and depth vector for overlapping objects, and negative results for non-overlapping objects. You don't need to test the actual algorithm which provides these results, just that the correct results are returned. You may test specific optimisations in the algorithm, but keep these separate from the main test code in case you change the algorithm later.

After you have written tests to cover the functionality in the code, then test edge cases (array lower and upper indexes, maximum and minimum numbers of sprites, level locations, inventory items, quest items). If you have combinations of items or quests, simulate testing them in all possible combinations using paramaterized tests in FlexUnit (use the Paramaterized test runner).

Performance is important with a game engine While FlexUnit does provide timings for each test, it comes with quite a large overhead. For this reason it is probably better to write your own framework for testing performance. It doesn't need to be complicated, it can be a simple class which calls a set of methods in order, measuring the time taken to run each function.

A useful tool to have is an automatic game-play script. Either hand-roll or record a game sequence, then play it back and verify the game state at each frame. This can be especially useful when optimising to measure speed increases.

The hardest code to test is visual and audio, as it requires some sort of signal processing to validate, and the resulting tests are usually fragile. Steer away from these initially, and do them last (if at all).

http://www.flexunit.org/?page_id=6

share|improve this answer
    
This is a nice answer. –  Magnus Wolffelt Aug 1 '11 at 15:04
    
Woah, very nice answer. :) Thanks for the good input! –  Baelnorn Aug 8 '11 at 13:15

Unit testing is hard. Since this is probably your first foray into testing, let me share some points with you from my years of experience:

  • Not everything is testable, nor easy to test, nor worth testing.
  • Tests, like code, need to be maintained and updated as the app changes.
  • If nobody is running tests, or keeping them up to date, you might as well not test.

Having said that, the easy, low-hanging fruit are your plain classes -- classes with simple methods and properties. Call methods and verify that the state of your classes are as expected -- that's the easiest unit testing.

Beyond this, it gets hard. Hudson/Maven integrate, but I'm not sure about with FlexUnit.

My experience with testing is that over time, you get things done faster because you have a safety net -- you can risk bigger changes because you know if it broken.

This is probably not the answer you want, but it's good advice-type information that I feel you need to know :)

share|improve this answer
    
Thanks for the pointers, any help is appreciated. :) –  Baelnorn Jul 10 '11 at 17:42
    
No worries. I hope someone can give you the answer you want; I'm not too experienced with TDD in Flash. –  ashes999 Jul 11 '11 at 17:51
1  
Disagree with unit testing being hard. It's all about mindset and perspective. And there are plenty of benefits to doing TDD and tests even if the tests are forgotten and never run again. Designing for testability is very similar to designing for extendability and maintainability. –  Magnus Wolffelt Aug 1 '11 at 15:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.