Simplest most effective way to rank and measure player skill in a multi-player environment?

Question

I know of the Truskill algorithm, it's quite complex but effective.

My question is there other algorithms/methods to determine a players skill for accurate measuring for multi-player competitive play?

Answer 1

I think asking for "simplest most effective" is an unrealistic requirement, but there are certainly a few good approaches. Rather than go into detail, I'll link to an article:

http://www.lifewithalacrity.com/2006/01/ranking_systems.html

This covers:

• ELO
• an ELO variant by Days of Wonder
• TrueSkill (and glicko, which it appears to be based on or at least similar to)
• eGenesis Ranking (for Tales in the Desert)

On the whole you'll see that they're all pretty much variations on the same theme - you pick a starting or average value for a player, and then the values are used to predict the game outcome. The difference between the actual outcome and the predicted outcome is used to modify the scores for each player, and the process repeats with the adjusted scores. Since each modification to the scores makes the predicted result more accurate, the scores converge on their 'true' values. (This actually assumes that it's a game of skill rather than chance, that the skill in question is capable of being linearly ranked, etc.)

Answer 2

What did you find complicated about TrueSkill? I thought it was a straightforward algorithm with just the right amount of knobs to tweak for different game modes, and it reduces to Elo when the game is head-to-head with high certainty.

If you're looking for the simplest way to to effectively rank and measure a player in a multiplayer environment - my recommendation is TrueSkill.

Answer 3

Depends on the game. There are a few problems you can get into:

• For games that have a luck component and a skill component (Bridge, Poker, Magic:the Gathering, etc.), most algorithms don't account for the fact that a weaker player can sometimes get lucky. If your game falls into this category, you'll need to do some work. Generally this means figuring out what percent is luck and what percent is skill (a difficult trick, but if you've been using a skill-based algorithm like Elo already, you could run some metrics on the results to figure out how often the algorithm predicts an upset versus how often it actually happens). Then you have to change the algorithm, and exactly what to change it to is probably beyond the scope of this question.

• For games where matches can be manipulated (I can choose to play a ranked game against my friend), you have to put extra safeguards in place to prevent players from purposefully throwing matches.

• For games where a player can get "out of practice" if they don't play regularly, the system could involve some sort of time-based degradation. The Glicko system is a mod of Elo that adds an "uncertainty" variable to each player's ranking, based on how many games they've played and how recently they played them; the more certain a player's ranking is, the less it changes from game to game.

• Multiplayer games (whether free-for-all, team-based, or some other player structure) need special care, of course. Some team-based games make it easier to figure out each individual's contribution, compared to others.

• Also ask what the purpose of your rating/ranking system is. In professional games and sports, the purpose is statistical: rating is used as a predictor of the outcome of any given match. The primary goal here is accuracy. However, this is rarely what players want; instead they want progression, a feeling they are getting better and climbing through the ranks (whether they are actually getting better or not). In short, there is a tradeoff between accuracy and fun that you need to consider.

Answer 4

What about a pairwise system like elo? It has been used for ages in "normal" sports with excellent results.

For free-for-all-matches you could interpret them as a bunch of pairwise matches and assign points based on the relative standings.