Is BDD (Behavior Driven Development) used in games?

Question

I have been reading about BDD - Behavior Driven Development for a while, and I find it really easy and usefull to convert features into code. BDD users often call it TDD done right.

BDD is a tool for software design, from outside to inside, from the bussiness value (or gameplay value) to code.

Dan North introducing BDD

Do you know any resources about BDD and Games other than this?

Answer 1

It's probably safe to say that BDD, like TDD, or (insert trendy development buzzword-paradigm here) is used by some game developers somewhere, but they probably don't know they are nor would they neccessarily be able to identify what BDD actually means. The question is really how much they use it and how much do they have to use it for it to matter to you?

For example, where I work, all our unit test names are "sentences" like Dan North suggests in that article you linked. That alone isn't sufficient to say that we use BDD, of course, but maybe it's what you really care about?

The focus, in my opinion, should not be on which buzzword you apply at a studio, but rather which productivity and development process techniques you employ overall. I find that the most productive teams are mixing and matching techniques from a variety of "buzzword-paradigms" rather than committing, dogmatically, to every bit of rigid doctrine some internet study says comprises one particular buzzword-paradigm.

I see this most often with the Agile trend: teams that identify themselves as "doing Agile" tend to be more inflexible (ironically) about the process than teams that are organically incorporating the bits of Agile that make sense for them. The former teams almost always end up being less productive, in my experience.

A development team is made up of humans, who are not interchangeable cogs in a machine. They operate uniquely as individuals and as the unique combination of themselves. The way to effective development is not to bend your humans into the {BDD, Agile, WhateverIsNext} mold but to be constantly re-assessing how the team is progressing and shoring up deficiences in the process, replacing broken techqniues, and reinforcing things that are working. In short, to focus on shipping a title and not on "being Agile (or whatever)."

Answer 2

I think BDD is appropriate in every in environment. As others mentioned you are developing software and as a result you should test it. I do like bdd for some of the random semantics mentioned like test names as sentences. I also like grouping certain tests together while still being able to test 1 class.

To combat other messages here I'd like to point out that on a larger project it is MUCH harder to refactor code without tests. If you refactor some code you are flying blind as to whether everything will explode in a blaze of glory or not. The tests help you catch things early. So you write your test, fail, code just enough to pass and continue. When you refactor you should do the same thing, but rather than write you revise the test. In most cases you run the test it will fail, you go change what you think should change and it STILL fails. At which point you realize that some other piece of code relies on this function/method a completely different way. You can then fix your test and the resulting code. Without that sort of code coverage you'd be stumbling around for days trying to find where stuff is broken, bugs are not easy to find or track, and moreover you REALLY can't refactor code and you start fearing your code base.

Go read about "Contracts" in the Pragmatic Progammer's book. Testing helps you achieve code contracts. This code does X and nothing more than X and don't expect it to do anything about Y or try to adapt it to do Z. It ensures code cleanliness and expects everyone to not be a dick and muddy up the code base.

There are more reasons to BDD. The main one for me is that I would do the same amount of testing to validate my assumptions anyways so I might as well formalize it.

On the point of "how" it really depends on your environment. I am writing a java game now and using robolectric. You should always attempt to "expect" something. I've heard that spies/mocks/stubs are not as useful since you need to have equivalent on the other side, but sometimes you have no choice especially with APIs. You can assume that the other side of the API is not terrible though and it is usually your code that sucks.

If for example you are testing movement. Well you expect when "Up" is pressed that the user moves forward by some measurement.

If for example you are testing graphics rendering... well don't test that too much because are you really doing that? A good test framework might handle this part for you. Reflection is not super trivial I'd say for these sorts of things. You may need to check buffers etc etc. I'd simply just check for what you are actually doing. Character is here, now he is there after some action.

You should have plenty of tiny little functions/tests and together they will sum up to something useful.

Answer 3

Is it? Maybe. My opinion would be that it would make for a very poor fit for entertainment software generally, although it might work well for the low level libraries.

EDIT: Here's some justification for my opinion.

Wikipedia defines BDD as a technique that "encourages collaboration between developers, QA and non-technical or business participants in a software project." This already sounds like a bad idea because games differ from most software in that they are not designed as tools to meet a specific need for a 'non-technical or business participant', but are cohesive works broadly designed to entertain. There is an emphasis on "desired software behaviour" but games rarely have 'desired software behaviour' except at the technical level. There is definitely merit in checking that part of the code, but not with the end user, because they will never see it.

But let's assume that you want to throw out that human stakeholder stuff and just use BDD to enforce contracts between different code modules, which as far as I can see doesn't differ much from normal test-driven development, which I also consider poorly-suited to games, for the following reason.

Tests are useful for checking that discrete events happened when expected. This works well in event-driven programming, ie. most of the software world, where an action is performed, some output is generated, and then you just verify that the action and the result match up. However, game software is typically a simulation, where an action does not have a discrete result but a continuous change in the world state. If my hidden player makes a noise, I might want to check that the AI hunts me down. So, I can create a test to make sure that the AI is in 'hunting' state after a noise is created, and that's great. But how do I know the hunting even works? You can't check that instantly - you can only observe it over the passage of time. You can hack in various tests to test certain aspects but this is not well-suited to a typical testing approach.

Additionally, a test-first approach can create a false sense of security, and lead people to believe code is better than it really is.

def check_dice_roll_in_range():
    d = new Dice()
    assert(d.roll() between 1 and 6)

class Dice:
    def roll():
        return 4

Since a test result can give a false positive, you can never escape the basic need to check the code itself. But if the code itself is checked adequately, the test takes on secondary importance. This is why, in my opinion, tests are best used after the event, to test bug fixes.

I wouldn't argue that there's never any benefit in testing that, when objects X and Y work together, the result you get is as expected. The issue is whether you are using the most effective way of verifying this. Methods could include formal verification, a code review, test-first methods, test-last methods, traditional QA black-box testing, or simply using the code as expected and observing the results. The last two options are surprisingly effective most of the time, because despite sounding like they lack rigour, most bugs are found during the course of typical use, and understanding a bug in its natural context can sometimes be easier than understanding it in an artificial test harness. On top of this, formal testing and verification often fails to find problems because (by definition) you can only test for the scenarios you expect - and it's not the expected scenarios that are typically the cause of poor software quality.

So, in summary, I think that test driven development is not necessarily a great choice for software, that tests alone are never sufficient to ensure software quality (and thus the time spent writing them must be compared against alternative uses of that developer time), that games are an especially poor match for automated test cases, and that games are an especially poor match for development methods that look to emphasis 'business value' and 'acceptance testing'.

(Hopefully that is a better answer, even if you don't agree with my points.)

Answer 4

I feel there's a misconception about what BDD is. BDD is not a TESTING technique or process. BDD is a development model and process. It goes WAY beyond testing and it goes way beyond programming.

As such, I don't know of any major AAA studio working with this model (I have friends working for some of them around the world as programmers). As someone else pointed out, it may be that some project manager or team somewhere incorporates some of the practices BDD encompass, but I don't know of any studio applying pure-BDD (from business value definition, to specification by example, to writing feature files, to validating them with stockholders, to automating the feature files as tests).