Are mathematical models a good way to test strategy games?

Question

I am currently working on a simple strategy game. The game is a space exploration game where the player has to try to keep the game going for as long as possible to score as many points as possible. To go from planet to planet the player needs to use resource points; each explored planet also gives the player points (decreases gradually with time), yet a small expense in resource points is also added for each explored planet. Therefore, the player has to keep exploring to avoid running out of points.

The knowledge points allow the player to try to discover more planets, before exploring them.

I am looking for a methodical way to test the game. At the moment I keep adjusting the formulas used for the points given by each planet, or the expenses, or the travel/exploration costs, yet this trial and error method is pretty tedious and not really efficient. At the moment my "strategy" game seems more like a "click as much as you can" kind of game.

Since each type of planet has its own advantages and inconveniences, I'd really like that to show in the game play.

I was thinking of using mathematical models in Excel, by inputting all my formulas and adjusting the parameters there, but the more I think about it the more I think I will have to use some "randomness" to reflect the player choices, which defeats the purpose of testing the game for its "strategic" aspect.

Here's a screenshot of the game:

Answer 1

Having a good understanding of your game mechanics is essential to balancing a game properly. Modeling your game as a set of equations and graphs is a great way to obtain that understanding.

Regarding randomness: Many random events balance out over time (law of large numbers), so when there is a 50% chance that X happens and a 50% chance that Y happens, you can often simply model this as half of the effect of either event. But player actions are not random! Any player who is not playing the game for the first time will not randomly click on buttons but pursue a certain strategy. So player behavior should be another variable in your model. In a 4x game you could model how the game will develop when the player focuses their resources on expansion, development, military or different splits between either. Even better when you can model different strategies in different phases of the game. This might tell you if there is an ideal strategy (in which case you need to guide the player towards it) or if there are many viable strategies with similar chances of success (in which case you need to encourage the player to experiment).

However, you also need to verify that your game model is correct. The only way to test that is through playtesting. Preferably by testers who are not yourself. As the developer you have certain preconceptions about how you want your game to be played, so you subconsciously play it like that. But real players have a completely different perspective and might play it in a completely different way.

So when your models predict that the player's KP income grows quadratically over the course of the game but in test games it grows more linearly, it's time to find out where your model is wrong. Or maybe the players just overlook some feature your game didn't make obvious enough? Or maybe KP income isn't as crucial to winning your game as you thought, so players have no reason to focus on that? Time to analyze test matches and interview the players.

Then you need to decide based on your tester's feedback what would make a better game: To change the game to better fit your model or leave the game as-is and change your model to better represent it? The best course of action can often be to do a bit of both.

But what if your model gets completely confirmed by the playtests but your players still didn't like it? Congratulations, you failed! Yes, that's a good thing, because the faster you fail the better. You can now go back to the drawing board, come up with a better game concept, update your spreadsheet models, implement them in your game, and start another round of playtesting.