8
\$\begingroup\$

There is many topics on how procedural generating works but I couldn't find any info on how to test and - what's more important - how to report bugs that may occur on procedurally generated levels. What is the best way to describe a bug, for example related to level design, on a map that will look different each time you play new game? What information should be included in bug report so reproducing it would be easier for level designer, level artist, etc. and then also during regression process?

EDIT: It is worth mentioning that the game, I am preparing to test, is in very early phase of developing and is constantly changing along with the systems responsible for generating levels. The purpose of this research is to learn how to describe bugs (what info to include in report) so the regression in a changing environment would be as less time consuming and as effective as possible.

\$\endgroup\$
  • 2
    \$\begingroup\$ Who's your target audience? I mean, who is going to fill in these bug reports? \$\endgroup\$ – Alexandre Vaillancourt Feb 15 '17 at 14:50
  • 1
    \$\begingroup\$ One of the possible solutions is to add "saving" functionality, or at least some hashing mechanism. That way the tester can include the whole level to his/her report or at least include textual hash that can be later reversed to a bunch of look-alike levels. The first option is straightforward but eats traffic/space. The second option needs some serious brainstorming to implement. P.s. I may be wrong, but as far as I remember Diablo 2 used hashing. So the level (and other levels that look more or less like it) was described by hash. \$\endgroup\$ – Sergey.quixoticaxis.Ivanov Feb 15 '17 at 15:01
  • \$\begingroup\$ What type of game? I.E. techniques used in a turn based game may be ill suited to a real-time game & vice versa. \$\endgroup\$ – Pikalek Feb 15 '17 at 15:21
  • \$\begingroup\$ @ Alexandre Vaillancourt: Testers will report bugs using an automated tool which will gather some simple information like the amount of physical objects, number of AI's etc. I'm wondering what else could be included not only in those automated reports but also in the conventional reports written by testers. @Pikalek: It's a FPP action game with an open world and real-time gameplay. \$\endgroup\$ – BoYLER Feb 16 '17 at 14:26
18
\$\begingroup\$

It can also be very useful when your map generation algorithm is deterministic and repeatable based on an initial seed value. So when you enter the same seed value, you get the same map. This might be easier to implement than you think. Most random number generation APIs can be initialized with a seed value and then always generate the same sequence of random numbers for the same seed. If you are using one of the few platforms where this is not the case (like browser-based JavaScript), you could implement your own random number generator. There are plenty of algorithms around which are not very difficult to implement and usually always require a seed value.

Your testers can then mention the map seed in their bug reports. A bug reports like "Trees grow in the middle of a lake" can have a reproduction instruction "Generate a map with seed a4f6dd32 and go to position 4321500:1269100". After you verified this, you can set a conditional debugger breakpoint in your map generation algorithm which triggers on the affected map chunk and check out exactly what's going on.

A deterministic map generation algorithm also allows you to implement automated test. You can have various test seeds with known outputs and then have your automated tests check if that output is still the same after you made some performance improvements in your generation algorithm.

\$\endgroup\$
  • \$\begingroup\$ Thank you all for the ideas. I cannot try and put them to use just yet, as the game is not in a playable/testable state.It actually will need a lot more time to be ready for testing. For now I am just doing research in this matter. \$\endgroup\$ – BoYLER Feb 16 '17 at 14:29
  • \$\begingroup\$ This is a great answer. I'm currently creating a game that uses Dungeon Architect to randomly generate levels based on a seed. I've gone a step further and created a random singleton class that doles out the Random values to ALL subsystems, both DA's and my own. The benefit is that if I know the seed value and the value of the level itself (say dungeon 1-1 or forest 3-6), I can recreate the exact same level, with the same mobs, in the same places with the same loot tables and everything. Since this value is logged, it would be easy for me to parse a bug report and recreate the scenario. \$\endgroup\$ – Jesse Williams May 24 '17 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.