How to measure skill in a pinball game?

Question

I am trying to come up with a system to measure or estimate the skill of a player at a single player game (so no head-to-head rating system help here). Specifically, it is a pinball game, but I do believe it is similar to most point-based single player games.

I have thought of 3 approaches:

• total points scored with a threshold for each skill level
• average points scored over all time
• average points scored over last X games

All of them use points, which I think is probably the right metric: anything a player does in pinball grant points, with the most difficult objectives worth more points. Points also take into account all of the rare combination of objectives (like score multiplier + multiball) that would be hard to track manually.

However, I am not sure which approach to use and I bet other games have implemented similar skill-measuring system that I can use as a guide, so I'd appreciate any and all help on this.

Answer 1

When you say you're going to measure skill by points, it's circular reasoning / logic. How are you going to decide how many points the player deserves for each goal accomplished? You can't rate the player's skill by points cause in order to do that, you require the base assumption that the sum of points accurately reflects the challenge surpassed, which may be false.

In order to properly judge a player's ability, you need to have some base data to compare with. You can collect this data by letting testers attempt different challenges; you can also let non-testers attempt these challenges and encourage them with badges.

Once you collect statistics from dozens of players or more (if possible), you could compute (an approximation of) the probability that a certain player will accomplish a certain feat. You could compare a new player to the statistics. You can even take the time played into account to tell if a person is relatively good for a beginner or relatively poor for an experienced player.

Answer 2

You would want to evaluate the player's skill using a set of recent data( three play-throughs) or something like that.

As a user continues to play, their skill increases along with their familiarity of the game mechanics.

Having a point threshold seems to be the best way to control a users skill. Naturally a person will have a couple good runs here and there that are outliers of thier base skill level. The trade off is that as you go back further, you are looking at the players previous skill level, so it is up to you to find the right amount of games to go back to balance the players skill level over a period of time, while accounting for their increase in skill over time.

Have you thought about classifying shots as "Skill shots" to see how many times a player hits a very limited or narrow shot per ball? This way you can eliminate some of the assumptions that evaluating a set of points would conclude.

Answer 3

I would think you could get an assessment of a player's skill by tracking a few things, and various skill levels could be defined based on these items. As stated already, overall score could be one of them. I think examining how well a player does on a per ball level, will also give you a good gauge to go by. For instance, how long a single ball is in play, how many times multipliers are achieved, how long the play maintains a multi-ball state, how many times hard-to-hit items are hit.

Again, a high score is an indicator of all these thing, however you might be able to factor out some dumb luck high scores by monitoring these items.