25
\$\begingroup\$

Context

I am continuing some legacy code for a game in Unity3d, and I want to write some functional tests meant for regression, to ensure I don't break things when implementing new things or when refactoring.

I already know there is the 'Unit Test Tools' suite for Unity3d available as an asset. I have used it as a unit-test suite, so I test my models (classes).

Example of what kind of test I am thinking about

For "functional test" I mean things like this:

  • Run the program
  • When in the menu scene, assert there is a button that says "start"
  • Click it
  • Then assert a new scene is loaded
  • Assert pixel XXX is red
  • Click in coordinate XXX
  • Assert now that the pixel changed to green
  • etc.

Questions

Q1: How do I write and run functional tests for my game? Is this also typically done in the UnityTestTools (UTT)?

Q2: If UTT is more for Unit-Testing, then is there a separate suite for functional testing? Which one?

Notes:

I'm targetting Android, and running Unity5.3.1f1

\$\endgroup\$
3
  • \$\begingroup\$ As far as i can tell the built in UTT is just for unit testing. There might be a testing tool that suits your needs in the asset store \$\endgroup\$ Feb 14, 2016 at 19:06
  • 3
    \$\begingroup\$ Actually, no. Unity Test Tools contain toolset for integration testing and plenty of assertions since the very first version. If you don't see anything like this in built-in UTT, then download full UTT from the Asset Store. \$\endgroup\$ Apr 11, 2016 at 13:05
  • \$\begingroup\$ Take a look at the Test Running documentation, Unity Test Runner Doc It allows both play mode and edit mode tests. \$\endgroup\$
    – Racksay
    Nov 1, 2017 at 9:13

2 Answers 2

1
\$\begingroup\$

As I understand it, you're looking to understand automation testing in Unity.

Unity Test Tools

This should be your goto method, but before digging into how, I'll examine the why.

The preferred way to do any form of automation testing is to do this in an environment as close to production as possible. Using something like the Unity Test Tools (living inside the Unity Editor) sounds like a bad idea for anyone who'd written conventional integration tests.

Why? Because with any integration testing, you want the least amount of extra stuff bogging down your application that's not there when the user opens it. The Unity Editor might have different performance or behavior compared to the iOS version (for example).

So, why use Unity Test Tools (UTT) then?

The main reason is convenience. Both the Unity Editor and UTT are built around being easy and visual to use. You'll have an easy time writing tests, hooking them with your specific game, and understanding when they fail.

What about the difference in behavior between the Unity Editor and production?

After all, the Unity Editor is a wrapper around your application. That means, there will be some differences between the Editor and production. But, integration testing aside, the Unity Editor has been a wrapper for a very long time now. It is a mature platform with thousands of game built on it. The results you get from running integration tests inside the Editor will be very accurate.

In the example scenario you provided, everything but the first item can easily be asserted using Unity Test Tools. You can open scenes and validate behaviors just like any other integration tests.

I suggest writing most (if not all) of your tests using UTT. To add compatibility with continuous integration tools (e.g. Jenkins), you might want to run those from the console using command line arguments.

Starting the application and manual testing

Starting the application isn't something you have precise control while in the Unity Editor. To validate that aspect of your game and more, you can use some basic manual tests.

Here's how to approach that:

  1. Make sure your game emits some kind of signal when it reaches the specific milestones you care (or might care) about. Logging to a file is the most common approach (using the automatic builtin logging or a 3rd party version).
  2. Periodically read and analyze the logs. This can happen manually or be scripted using your second-favorite scripting language.

Alternatives

Not surprisingly, as everything Unity related, we have more than a few alternatives. Here are some you might want to consider:

Depending on the scale and scope of your project you might want to employ one or many of the options I outlined above.

\$\endgroup\$
0
\$\begingroup\$

For anyone finding this question post 2019 this is now possible with the Unity Test Framework. https://docs.unity3d.com/Manual/testing-editortestsrunner.html

\$\endgroup\$
1
  • \$\begingroup\$ Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. \$\endgroup\$
    – Glorfindel
    May 17, 2020 at 20:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .