Developing an ELO like point system for a multiplayer gaming site

Question

I'm currently working on a gaming site where users will submit virtual players for different games, like Chess, Nash, Backgammon, Go, etc. The idea is that users don't compete themselves, but through their virtual players. There will be leagues, tournaments, and other competition formats.

The question is which would be a good rating system for users in this environment. Take into account that every user may have many different virtual players playing in many different games. As a general guideline I would like to guarantee the following properties:

• Users who have a lot of mediocre players should not score higher than users with a few very good players.
• A user with a high rating should not be penalized if he adds a new bad player, until he has had enough time to improve his player.
• Users who don't play often should not score higher than users who play every day.

Thanks in advance.

Answer 1

I would divide the ranking into 3 levels:

1. Bottom level is where players are rated. Use ELO or a similar system, where a strong player is penalized much more for loosing with a weak opponent, than he could be awarded. Also secure game system for users using many accounts, e.g. by random matchmaking. This will solve your 3rd point:

   • Users who don't play often should not score higher than users who play every day.

2. Middle level is rating for every group of players owned by one user and playing one game, e.g. Alejandro's Chess players. You most probably don't want a chess player to play with a Go player, as they use different algorithms, and even the best player of one game will probably lose with one of the worst players in another. To calculate the rating of a group, take an average of ELO of the players in the group, not considering 20% of the worst players in this group. This will ensure the extraordinarily terrible players won't affect average rating. Keep in mind this is yet another reason to make matchmaking random. So this covers two remaining points of your requirements

   • Users who have a lot of mediocre players should not score higher than users with a few very good players.
   • A user with a high rating should not be penalized if he adds a new bad player, until he has had enough time to improve his player.

3. Top level is rating for a user. This is a very subjective part; you could for example:

   • show best achievement of the user (#23 place in silver league)
   • show average of scores of all game-groups
   • show average of scores of all game-groups the user is active in
   • show the highest of scores of game-groups
   • calculate the main score in a complex way, e.g. find the highest possible score of a user, being an average of chosen groups plus 100 bonus points for every group taken into an average.

Answer 2

ELO works by matching you with an opponent based on your rating, and then transferring some rating from one player to the other based on the result and the rating difference between the players (i.e. a high rated player beating a low rated player gains very little, but if he lost the game he would lose a lot).

I think that is not going to work well without a separate rating per game for each player. For example if a really good backgammon player decided to learn chess, it would seem strange for their rating to decrease while they are just learning how to play. You'd also need to be able to match them with a chess opponent of the appropriate skill, which a single overall rating can't do effectively.

You could still have an overall rating which is calculated as say the maximum of all other ratings, or maybe the sum of all other ratings if you want to encourage people to play all the games.

I would also strongly consider only allowing one player per user, or alternatively one player per game type. This is because new players need to be awarded an initial ELO rating. If they lose their first game then that rating will drop, and creating a new user and getting a fresh rating would be an easy way to exploit the system.

Answer 3

It looks like you have three free variables to play with. The first one is the number of 'mediocre playes versus better players', the second one is 'user rating agains new player success rate' and the third one is 'frequent players versus infrequent players'.

My suggestion: Divide and conquer. Find an elo-inspred rating for each of the three variables and take a (weighted) average of it. Now you have three problems, but the sum of problems with the three might be smaller than the problems of the whole.

Answer 4

I've thought about ranking systems quite a bit and I think it's best to create a new ranking system whenever you have a new project since every time you'll have different parameters and requirements. You could adjust for example parameters of the elo ranking etc. but this will always result in a half-hearted solution. Most game companies create their own rankings & matchmakings for every game.

Of course the already existing rankings are not bad by any means but usually they try to generalize a given scenario which only gives a rough trend. Looking at how other ranking systems work is a good basis for your own ranking but usually creating a ranking system is a complex process which even after implementation requires a lot of tweaking. Thousands or millions of dummy scenarios are required in order to evaluate the quality of a ranking system and often the math behind it is not that simple.

Writing down how the ranking is supposed to work is one of the first steps at creating one, although this might sound simple it is a lot more complex than you might think.

Example:

Users who don't play often should not score higher than users who play every day.

This sounds obvious but on the other hand, how do you quantify often? Is someone who only played one time (scored high points) supposed to have a higher ranking who has mid scores but plays every day? You would have to define a mathematical relation between amount of games/days played with the amount of points. How does this relation look like? Is it supposed to be linear, logarithmic, exponential etc. How will this effect the strategy of the players (will they play less put try to play well in order to get higher points)? Will this behavior have an impact on other aspects of the game (less trained players per user?)?

Every requirement will affect the ranking and thus you have to analyse how they will effect each other (maybe give some higher priorities than others or even ignore certain requirements in some cases). Every ranking system is comparable to a dissertation (depending on how detailed you want to make it)

Ranking

I'm not sure whether I understood your request correctly but to me it sounds like you want to create a ranking for every user based on the success of every player a user has.

My first idea would be to create a ranking for every game (chess, go etc.). Rank every player who plays this game in it and then calculate the overall ranking based on these rankings for every user. Of course this would be very complex since you would have to create a ranking for every game BUT there are lot's of decent rankings for every game. This way the only thing you would have to think about (given you found a ranking for every game) is in which way you want to calculate the overall ranking based on the rankings of every game (still not easy but way easier than creating only one ranking containing all information of every game).

This is what I would recommend to someone who doesn't want to spend too much time with the math behind it, which is still a lot.

Edit:

The advantage of a ranking for every game would be that you could use this ranking for your matchmaking process, but this is another topic.

Answer 5

Before diving into formulae, if you want to create a ranking system, you'll have to come up with a list of specifications and requirements. Decide what the ranking system should accomplish and work from there. Be specific. Requirements that are rock solid can be a great guide, where general guidelines, such as the ones you drew up, may lead to confusion instead.

An example. Alice and Bob both have a few good virtual players, but Alice's are slightly better. Alice then continues to add mediocre players to her set. Eventually, she'll have quite a lot of them. According to rule #1, Alice's score may no longer be higher than Bob's and therefore must go down. Rule #2 forbids this.

Carol joins the party. She has one virtual player and it's terrible, no match for anyone of Bob's players. However, on account of her enthousiastic daily participation and rule #3, she still scores higher than Bob, who only plays once a week.

If you stick to basic limitations instead, even seemingly trivial ones like 'winning increases your score', 'playing often adds to your score'... as long as the set is consistent, it will make the mathematics a lot easier.

Answer 6

Some function to test and to fool around with for gaining/loosing rating points.

Say player A has rating A and player B has rating B.

The difference in rating is: D = |A - B|

Put a cap on the difference: if (D > Dmax) D = Dmax

You need some function to calculate the gain/loss for a match.

AdjustRatings(Player Winner, Player Looser)
{
    const int Dmax    = 100;
    const int MaxGain =  50;

    int Difference = Looser.elo - Winner.elo;
    if ((Difference) < -Dmax) Difference = -Dmax;
    else if (Difference > Dmax) Difference = Dmax;

    // the gain is a ratio from the differecene in rating.
    int Gain = (Dmax + Difference) * MaxGain/DMax;

    Winner.elo += Gain * Winner.FrequencyFactor;
    Looser.elo -= Gain * Looser.FrequencyFactor;
}