Hot answers tagged testing
46
One-person independent game. It was a multiplayer tank game with destructible terrain, and the destructible terrain and collision code proved somewhat flaky.
I ended up rigging up some basic dumb AIs (by "dumb", I mean "absolutely idiotic" - they would randomly choose "drive towards an enemy tank", "drive away from an enemy tank", and "drive in a random ...
21
For an MMO I worked on (100ish developers, PC focused), we tried to add a huge variety of automated testing with varying success. Here's what worked:
Basic tests during our automated build process were a huge win. This included tasks such as creating a character, transferring maps, running some scriptable UI tests and looking for expected behavior. This ...
20
Two words: headless client.
Basically it's an AI driven client that's had the all graphics bypassed so that you can run many multiples of copies on a machine, you can even run them on servers because you never initialize the graphics at all.
Above all those clients is a supervisor that is responsible for starting, monitoring and stopping all the clients to ...
15
Two big helps in difficulty tuning.
Metrics
Tracking player data can go a long way to making difficult tuning as objective a process as possible. How long are people staying alive, where are the taking damage or getting killed, how many tries does a section of the level take. Getting good metric data can really help your team see where people are having ...
14
Working on a 4x strategy game with 3d combat (think Homeworld meets Masters Of Orion) that unfortunately never saw the light of day as the company ran out of funding..
I always ensured that you could play the game without human players so we could leave the game running overnight.
We could turn off the 3d combat (simplified to a random result) and we left ...
12
It's probably safe to say that BDD, like TDD, or (insert trendy development buzzword-paradigm here) is used by some game developers somewhere, but they probably don't know they are nor would they neccessarily be able to identify what BDD actually means. The question is really how much they use it and how much do they have to use it for it to matter to you?
...
11
I don't think TDD, as such, is appropriate as a foundation for game development. Automated unit testing as part of methodology, sure, but too many of the key concerns of game development are subjective and not machine-testable for testing to be the driver of development. How are you going to write a scripted test for whether a game mechanic is fun? That a ...
11
There is software available to perform epilepsy testing as defined by OFCOM, known as the Harding Test.
Unfortunately, the software seems to be commercial only and requires a data capture card for it to run, so it's not the most financially feasible solution for an indie dev. This can be found here: http://www.hardingfpa.tv/
However, there is an ...
11
For traffic load testing there are many tools, but you're going to need to test other things as well that will be specific to your game server because network traffic is only one part of the equation -- the processor utilization and constant database queries are two others that immediately come to mind as important factors that you'll want to measure (there ...
10
A sort of code coverage approach to test cases can be done using simple flags that are tripped once the block of code has been executed in-game. Displaying which flags have been tripped and untripped on screen can allow testers to know which cases have been covered and which haven't.
Simple as it is, it's still effective so long as the flags have ...
10
Check:
- How much RAM it uses
- How much texture + GPGPU memory it uses
- Minimum frame rate / Current frame rate * CPU speed * SpeedUpFactorOf(minimal number of cores you can use)
- Similar equation for GPU calculation units
- what level of opengl/directx/cuda your game engine requires from the graphics adaptor
- how much disk space the installation + ...
9
You should have some rough idea of the minspec just by developing your game -- for example if you made an explicit choice to use D3D10+, you obviously are culling out XP machines. So that's a place to start.
Find a way to test on as many machines as possible, ideally after implementing some way to gather performance metrics automatically (as users can be ...
9
It's certainly viable, although a lot of game programmers haven't really gotten on board with the idea yet, or have a good understanding of how to test complicated systems. I admit myself that I rarely use it, except for non-gameplay-related systems that are easy to test.
Expect to use a lot of mock objects. Because of how tied together a lot of systems ...
8
On a first person shooter I worked on (Descent 3 -- linux/mac/windows, ~30 people on the team in 1999), the demo recording/playback capability turned out to be extremely useful. I made an option where you could playback the demo as fast as the game could render frames, and that became a great way to verify performance after a bunch of things changed.
It ...
8
Watch someone play but don't talk to them. By not talking to them, you can see things them do things that will make you pull you hair (You: the solution is obvious. What's wrong with you? #$@!).
Instead of saying your thoughts aloud, write down what you want to say. Use this as your basis to guide your game difficulty.
8
I would strongly suggest having an in-game bug-filing option, preferably a big button that says "FILE BUG HERE". Players who are engaged, especially if your game is actually good, are never going to remember bugs that occurred if they have to go out of the engine to file them. You could have those go to the bug tracker, or you could just have it email a ...
8
Best advice I could give is to put what you have up, making it clear that it is still a beta and that you want people to play test it and link to it everywhere. This includes every game site you can think of include gamedev.net, moddb, here, facebook, twitter.
There is nothing wrong with advertising mate. How else are you going to get people to play your ...
8
I wrote a tip/trick on CodeProject that discusses this very problem.
Samsung are essentially providing free access to actual devices via the web over here, I'm not sure if the same thing is provided by any other vendors though.
7
A few things I would say are important:
Encourage Programmer Unit Testing
This will make sure that certain stupid bugs, if there is a unit test for them, won't recur, because the unit test will fail if they do. This requires a change in programming methodology, but in my opinion it is completely worth it.
Automate Whatever Testing You Can
Beyond unit ...
7
This seems like a good application of the Approval Tests framework or something like it.
As stated in the comments, you are still going to have an issue with false positives, if you happen to approve bad output, but this will at LEAST tell you when output has changed significantly.
Since you are using OpenGL, I'm assuming that approvals won't work for you ...
7
I think you are worried about the wrong things Milo. I see from your other questions you are planning on writing the game is C++.
Even a not very well coded sever written in a scripting language would be able to handle 100's if not 1000's of requests with out apparent lag in a card game.
These kinds of technical details are probably not going to be a ...
7
The problem with this approach is in this statement -
"Add some more random chars, emulating new players"
The task of randomly generating every permutation of character builds is trivial compared to making the AI that would actually use that build appropriately. It would be extremely difficult to tell the difference between a poor build and a poor AI in ...
6
I found UnitTest++ to be very easy to work with.
I will yet have to try amop alongside with it, which was mentioned to be a good companion to UnitTest++ for mock object functionality. Otherwise Google Mock is a popular choice.
Also, you may want to read up on UnitTest++ and Mock Objects.
UnitTest++ can be set up with your Continuous Integration approach, ...
6
If you are playtesting early builds, it's good to test in-person. You can actually observe players getting frustrated, discovering exploits, or having fun in real-time, and it's easier to ask them questions immediately after they are done testing. For indie games, simpler solutions are often better.
Put your game on a laptop and go to a place where people ...
6
It's a bit difficult to work out exactly what you're asking since the title is purely about TDD but you seem to be making Scrum an integral part of the question too - and as I understand it, one does not require the other.
If I believe this GD question, TDD is not much of a use in game development.
That is correct. I don't think I've ever heard of it ...
6
You cannot make sure that your game will not trigger seizures, ever, since the thresholds differ by orders of magnitudes between individuals and beetween the same person on different days and under influence of a hundred other factors which you cannot know about (lighting conditions, distance to the display, sleep deprivation, stress level, background noise, ...
6
Testing/QA in the games industry is basically just like anywhere else, albeit with a tendency towards less process and formal methodology than in non-game work. Companies will hire for full-time QA as well as contract staff; the ratio of each will vary based on the culture of the studio and the needs of their current project or projects (typically contract ...
6
You'll have to do some market research, dirty words I know in tech circles, but there it is =)
Have you personally, or your design staff anyways, ever played any free to play titles? And by played I mean having spent a couple months on both sides of the "I paid for some things" divide.
Find all the FtoP games you can, not limited to your genre but not ...
6
No simulators that I know of give reasonable expectations of on-device performance. The iOS simulator, in particular, is very bad. And I think the Android one is an emulator.
As to what devices you should support, that's really a business decision that you have to make on your own based on the theoretical cost/benefit of making your particular game work ...
5
Basically you want to test on as many other machines as you can - from your friend's hard-core gaming rig to your gran's email-and-internet-only machine. Especially your gran's email-and-internet-only machine, because it's probably loaded with cruftware and other stuff that will interfere and slow your game to a crawl.
PC compatibility testing is a big ...
Only top voted, non community-wiki answers of a minimum length are eligible
